Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: Add workflows to push, pull, and validate docker images #1676

Merged
merged 34 commits into from
Jul 25, 2023
Merged
Show file tree
Hide file tree
Changes from 32 commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
f395322
add GitHub Action workflow for Docker deploy
nasdf Jul 20, 2023
9ddc073
temporarily run on push
nasdf Jul 21, 2023
96881df
update workflow triggers
nasdf Jul 21, 2023
5e2bdfc
temporarily use pull_request_target in workflow
nasdf Jul 21, 2023
d5569e7
add types edited to pull_request_target
nasdf Jul 21, 2023
fcce941
update workflow triggers
nasdf Jul 21, 2023
7ee2a85
Merge branch 'develop' into nasdf/feat/ci-docker-deploy
nasdf Jul 21, 2023
495f6a6
revert to pull_request trigger
nasdf Jul 21, 2023
3fdf241
use file instead of path in docker/build-push-action step
nasdf Jul 21, 2023
0c4f58a
remove pull_request workflow trigger
nasdf Jul 21, 2023
6084321
remove push branch trigger
nasdf Jul 21, 2023
10bf160
workflow fixes
nasdf Jul 21, 2023
67d06f5
Merge branch 'develop' into nasdf/feat/ci-docker-deploy
nasdf Jul 24, 2023
ab98758
add wasmer build step to docker file
nasdf Jul 24, 2023
0386984
update workflow to test Docker image
nasdf Jul 24, 2023
841f0b8
remove pull_request workflow trigger
nasdf Jul 24, 2023
ceb8374
remove wasmer build from Dockerfile
nasdf Jul 24, 2023
735dd59
remove pull_request trigger
nasdf Jul 24, 2023
cd04f43
Merge branch 'develop' into nasdf/feat/ci-docker-deploy
nasdf Jul 24, 2023
766070f
add pull-docker-image and validate-containerfile workflows
nasdf Jul 24, 2023
a2a9c33
build docker for amd64 and arm64. update workflow to test both arm64 …
nasdf Jul 24, 2023
323a3bd
use buildx in docker workflows
nasdf Jul 24, 2023
d20cf4e
remove docker test from push workflow
nasdf Jul 24, 2023
78341c0
use matrix for docker test workflows
nasdf Jul 25, 2023
000e4f0
simplify pull docker test matrix
nasdf Jul 25, 2023
75926f7
remove unused env from push-docker-image-to-registries workflow
nasdf Jul 25, 2023
45ef91f
add workflow dispatch
nasdf Jul 25, 2023
3d24a72
revert Docker platforms workflows
nasdf Jul 25, 2023
6554ba6
trigger pull-docker-image on pr
nasdf Jul 25, 2023
a6e8f89
update pull-docker-image tag
nasdf Jul 25, 2023
4eab4d2
remove test workflow triggers
nasdf Jul 25, 2023
e2054de
Merge branch 'develop' into nasdf/feat/ci-docker-deploy
nasdf Jul 25, 2023
3ab9289
set push-docker workflow trigger to push
nasdf Jul 25, 2023
d30d761
fix push-docker-image-to-registries job name
nasdf Jul 25, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 51 additions & 0 deletions .github/workflows/pull-docker-image.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
# Copyright 2023 Democratized Data Foundation
#
# Use of this software is governed by the Business Source License
# included in the file licenses/BSL.txt.
#
# As of the Change Date specified in that file, in accordance with
# the Business Source License, use of this software will be governed
# by the Apache License, Version 2.0, included in the file
# licenses/APL.txt.

# This workflow validates that the images pushed to the container
# registries can be pulled then run sucessfully.
name: Pull Docker Image Workflow

on:
workflow_run:
# Warning: this workflow must NOT:
# - interact with any new code.
# - checkout new code.
# - build/compile anything (only pull).
# - make any indirect calls (i.e. make xyz, or npm install, etc.)
# Note this workflow:
# - will use the base's (or default) workflow file's state.
# - doesn't run on the PR or the branch coming in, it runs on the default branch.
# - has read-write repo token
# - has access to secrets
workflows: ["Push Docker Image To Registries Workflow"]
types:
- completed

jobs:
pull-docker-image:
name: Pull docker image job

if: ${{ github.event.workflow_run.conclusion == 'success' }}

runs-on: ubuntu-latest

strategy:
fail-fast: false
matrix:
image_tag:
- sourcenetwork/defradb:latest
- ghcr.io/sourcenetwork/defradb:latest

steps:
- name: Pull Docker image
run: docker pull ${{ matrix.image_tag }}

- name: Test Docker image
run: docker run --rm ${{ matrix.image_tag }}
83 changes: 83 additions & 0 deletions .github/workflows/push-docker-image-to-registries.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
# Copyright 2023 Democratized Data Foundation
#
# Use of this software is governed by the Business Source License
# included in the file licenses/BSL.txt.
#
# As of the Change Date specified in that file, in accordance with
# the Business Source License, use of this software will be governed
# by the Apache License, Version 2.0, included in the file
# licenses/APL.txt.

# This workflow builds a Docker container image, if the build is successful
# then it will deploy the image to DockerHub & GitHub container registries.
name: Push Docker Image To Registries Workflow

on:
push:
tags:
- 'v[0-9]+.[0-9]+.[0-9]+'

env:
TEST_TAG: sourcenetwork/defradb:test

jobs:
push-to-registries:
nasdf marked this conversation as resolved.
Show resolved Hide resolved
name: Push Docker image to registries job

runs-on: ubuntu-latest

permissions:
packages: write
contents: read

steps:
- name: Check out the repo
uses: actions/checkout@v3

- name: Set up QEMU
uses: docker/setup-qemu-action@v2

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2

- name: Build Docker image
uses: docker/build-push-action@v4
with:
context: .
file: tools/defradb.containerfile
load: true
nasdf marked this conversation as resolved.
Show resolved Hide resolved
tags: ${{ env.TEST_TAG }}
labels: ${{ steps.meta.outputs.labels }}

- name: Test Docker image
run: docker run --rm ${{ env.TEST_TAG }}

- name: Log in to Docker Hub
uses: docker/login-action@v2
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}

- name: Log in to the Container registry
uses: docker/login-action@v2
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@v4
with:
images: |
sourcenetwork/defradb
ghcr.io/${{ github.repository }}

- name: Push Docker images
uses: docker/build-push-action@v4
with:
context: .
file: tools/defradb.containerfile
push: true
nasdf marked this conversation as resolved.
Show resolved Hide resolved
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
58 changes: 58 additions & 0 deletions .github/workflows/validate-containerfile.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
# Copyright 2023 Democratized Data Foundation
#
# Use of this software is governed by the Business Source License
# included in the file licenses/BSL.txt.
#
# As of the Change Date specified in that file, in accordance with
# the Business Source License, use of this software will be governed
# by the Apache License, Version 2.0, included in the file
# licenses/APL.txt.

# This workflow tests that the container build is successful and
# that the built container runs successfully.
name: Validate Containerfile Workflow

on:
pull_request:
branches:
- master
- develop
paths:
- '.github/workflows/validate-containerfile.yml'
- 'tools/defradb.containerfile'

push:
branches:
- master
- develop

env:
TEST_TAG: sourcenetwork/defradb:test

jobs:
validate-containerfile:
name: Validate containerfile job

runs-on: ubuntu-latest

steps:
- name: Check out the repo
uses: actions/checkout@v3

- name: Set up QEMU
uses: docker/setup-qemu-action@v2

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2

- name: Build Docker image
uses: docker/build-push-action@v4
with:
context: .
file: tools/defradb.containerfile
load: true
tags: ${{ env.TEST_TAG }}

- name: Test Docker image
run: docker run --rm ${{ env.TEST_TAG }}

7 changes: 6 additions & 1 deletion tools/defradb.containerfile
Original file line number Diff line number Diff line change
Expand Up @@ -9,11 +9,16 @@ WORKDIR /repo/
COPY go.mod go.sum Makefile ./
RUN make deps:modules
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thought: Maybe we can get by not building lens deps here as they might be only dev dependencies. I will make a ticket and it can be taken care of outside the scope of this PR.
New Issue here:
#1707

COPY . .
# manually copy libwasmer.so to fix linking issue https://github.com/wasmerio/wasmer-go/issues/281
RUN export WASMER_ARCH=$(go env GOHOSTARCH | sed "s/arm64/aarch64/") && \
export WASMER_PATH=$(go env GOMODCACHE)/github.com/wasmerio/[email protected]/wasmer/packaged/lib/linux-$WASMER_ARCH/libwasmer.so && \
cp $WASMER_PATH /lib/libwasmer.so
RUN make build

# Stage: RUN
FROM gcr.io/distroless/base-debian11
FROM debian:bookworm-slim
COPY --from=BUILD /repo/build/defradb /defradb
COPY --from=BUILD /lib/libwasmer.so /lib/libwasmer.so

# Documents which ports are normally used.
# To publish the ports: `docker run -p 9181:9181` ...
Expand Down