Skip to content

Add release workflow #12

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
cpretzer merged 8 commits into from
Jun 30, 2020
Merged

Add release workflow #12

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
edbd1af
Add release workflow
aliariff Jun 9, 2020
41f3021
Use SHA for reference
aliariff Jun 23, 2020
4055e1d
DRY buildx command
aliariff Jun 24, 2020
da3fcd2
More specific version info
aliariff Jun 24, 2020
7067131
Add builder target
aliariff Jun 24, 2020
2e6fa28
Install QEMU together with setup builder
aliariff Jun 24, 2020
3edcd68
Add more description
aliariff Jun 24, 2020
4b7f804
More detailed description
aliariff Jun 24, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/integration_tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ jobs:
runs-on: ubuntu-18.04
steps:
- name: Checkout code
# actions/checkout@v2
# actions/checkout@v2.0.0
uses: actions/checkout@722adc6
- name: Setup KinD
# engineerd/setup-kind@v0.4.0
Expand Down
29 changes: 29 additions & 0 deletions .github/workflows/release.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
name: Release
on:
push:
tags:
- "*"
jobs:
buildx:
name: Build & Push Multi Arch Images
runs-on: ubuntu-18.04
steps:
- name: Checkout code
# actions/checkout@v2.0.0
uses: actions/checkout@722adc6
Comment thread
ihcsim marked this conversation as resolved.
- name: Configure gcloud
# linkerd/linkerd2-action-gcloud@v1.0.1
uses: linkerd/linkerd2-action-gcloud@308c4df
with:
cloud_sdk_service_account_key: ${{ secrets.CLOUD_SDK_SERVICE_ACCOUNT_KEY }}
gcp_project: ${{ secrets.GCP_PROJECT }}
gcp_zone: ${{ secrets.GCP_ZONE }}
- name: Set up Docker Buildx
# crazy-max/ghaction-docker-buildx@v3.1.0
uses: crazy-max/ghaction-docker-buildx@373dafb
- name: Docker Buildx (build)
run: make images
- name: Docker Buildx (push)
run: make push
- name: Docker Check Manifest
run: make inspect-manifest
4 changes: 2 additions & 2 deletions .github/workflows/static_checks.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ jobs:
image: golang:1.13.4
steps:
- name: Checkout code
# actions/checkout@v2
# actions/checkout@v2.0.0
uses: actions/checkout@722adc6
- name: Lint
# golangci/golangci-lint-action@v1.2.1
Expand All @@ -29,7 +29,7 @@ jobs:
image: golang:1.13.4
steps:
- name: Checkout code
# actions/checkout@v2
# actions/checkout@v2.0.0
uses: actions/checkout@722adc6
- name: Format
run: make fmt
2 changes: 1 addition & 1 deletion .github/workflows/unit_tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ jobs:
image: golang:1.13.4
steps:
- name: Checkout code
# actions/checkout@v2
# actions/checkout@v2.0.0
uses: actions/checkout@722adc6
- name: Run unit tests
run: make test
14 changes: 11 additions & 3 deletions Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,19 @@
## compile proxy-init utility
FROM golang:1.12.9 as golang
FROM --platform=$BUILDPLATFORM golang:1.12.9 as golang
Comment thread
ihcsim marked this conversation as resolved.
WORKDIR /build

# cache dependencies
COPY go.mod .
COPY go.sum .
RUN go mod download

# build
COPY . .
RUN CGO_ENABLED=0 GOOS=linux go build -o /out/linkerd2-proxy-init -mod=readonly -ldflags "-s -w" -v
ARG TARGETARCH
RUN CGO_ENABLED=0 GOOS=linux GOARCH=$TARGETARCH go build -o /out/linkerd2-proxy-init -mod=readonly -ldflags "-s -w" -v

## package runtime
FROM debian:stretch-20190812-slim
FROM --platform=$TARGETPLATFORM debian:stretch-20190812-slim
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
iptables \
Expand Down
29 changes: 28 additions & 1 deletion Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
DOCKER_REGISTRY ?= gcr.io/linkerd-io
REPO = $(DOCKER_REGISTRY)/proxy-init
TESTER_REPO = buoyantio/iptables-tester
VERSION ?= $(shell git describe --exact-match --tags 2> /dev/null || git rev-parse --short HEAD)
SUPPORTED_ARCHS = linux/amd64,linux/arm64,linux/arm/v7
PUSH_IMAGE ?= false

.DEFAULT_GOAL := help

Expand Down Expand Up @@ -39,7 +42,7 @@ integration-test: image ## Perform integration test
###############
.PHONY: image
image: ## Build docker image for the project
docker build -t $(REPO):latest .
DOCKER_BUILDKIT=1 docker build -t $(REPO):latest .

.PHONY: tester-image
tester-image: ## Build docker image for the tester component
Expand All @@ -49,3 +52,27 @@ tester-image: ## Build docker image for the tester component
kind-load: image tester-image ## Load the required image to KinD cluster
kind load docker-image $(REPO):latest
kind load docker-image $(TESTER_REPO):v1

.PHONY: images
images: ## Build multi arch docker images for the project
docker buildx build \
--platform $(SUPPORTED_ARCHS) \
--output "type=image,push=$(PUSH_IMAGE)" \
--tag $(REPO):$(VERSION) \
--tag $(REPO):latest \
Comment on lines +57 to +62

@ihcsim ihcsim Jun 24, 2020 •
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(@cpretzer @olix0r LMKWYT.)

I think this only works OOB with Docker Desktop, right? Specifically, it only works with a Buildx builder that uses the docker-container driver. On my Ubuntu laptop though, the default builder uses the docker driver, which doesn't support multi-platform builds.

Assuming that we want the Makefile to work outside of Docker Desktop and CI too, we should add some instructions like the following to the README:

On some environments where the default Buildx builder doesn't use the docker-container driver, the make images target may return the following error:

$ make images
multiple platforms feature is currently not supported for docker driver. Please switch to a different driver (eg. "docker buildx create --use")
Makefile:57: recipe for target 'images' failed
make: *** [images] Error 1

Use the following commands to create a builder that uses the docker-container driver:

$ docker run --rm --privileged multiarch/qemu-user-static --reset -p yes

$ docker buildx create --name=multiarch --driver=docker-container --platform=linux/amd64,linux/arm64,linux/arm/v7 --use
multiarch

$ docker buildx inspect multiarch --bootstrap

@aliariff aliariff Jun 24, 2020

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, instructions in the readme or directly in make help about how to build multi-arch images will be helpful.

@ihcsim ihcsim Jun 24, 2020

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can script it as a different make target (say, make builder), and mention in the README that if one encounters the above error message, run make builder. What do you think of something like:

.PHONY: builder
builder:
   docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
   docker buildx create --name=multiarch-builder --driver=docker-container --platform="${SUPPORTED_ARCHS}" --use
multiarch
   docker buildx inspect multiarch-builder --bootstrap

@aliariff aliariff Jun 24, 2020

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is also possible, I will try it out.

@aliariff aliariff Jun 24, 2020

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ihcsim
is it necessary to run docker run --rm --privileged multiarch/qemu-user-static --reset -p yes?
because in this case, we don't use QEMU, we just utilize golang internal cross-compile features.

@ihcsim ihcsim Jun 24, 2020

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AIUI, if the Dockerfile has the RUN commands, we will need the qemu in order for those commands to be executed during the multi-arch build.

@ihcsim ihcsim Jun 24, 2020 •
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AFAICT, the buildx github action has to do the same thing. See https://github.com/crazy-max/ghaction-docker-buildx/blob/44fd46c7833e095f13bb1d8ace66bd821198bfef/src/main.ts#L24-L38.

.

.PHONY: push
push: ## Push multi arch docker images to the registry
PUSH_IMAGE=true make images

.PHONY: inspect-manifest
inspect-manifest: ## Check the resulting images supported architecture
docker run --rm mplatform/mquery $(REPO):$(VERSION)
docker run --rm mplatform/mquery $(REPO):latest

.PHONY: builder
builder: ## Create the Buildx builder instance
docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
docker buildx create --name=multiarch-builder --driver=docker-container --platform="$(SUPPORTED_ARCHS)" --use
docker buildx inspect multiarch-builder --bootstrap
8 changes: 8 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,3 +19,11 @@ Then run the tests with:
```bash
make integration-test
```

# Build Multi-Architecture Docker Images with Buildx

Please refer to https://docs.docker.com/buildx/working-with-buildx/ to enable Buildx.

Run `make builder` to create Buildx instance before starting to build the images.

Run `make images` to start build the images.

@ihcsim ihcsim Jun 24, 2020

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding this section. Can we provide more context around why and when a user will need to run make builder? How about something like:

In some local environments like Ubuntu, where the default Buildx builder uses the docker driver, the make images command might fail with the following error:

$ make images
multiple platforms feature is currently not supported for docker driver. Please switch to a different driver (eg. "docker buildx create --use")
Makefile:57: recipe for target 'images' failed
make: *** [images] Error 1

To fix this, you can create a new Buildx builder instance by running make builder. This command will create a builder that uses the docker-container driver that can build multi-platform images. For more information, see the Buildx builder documentation.