Skip to content

Commit 0f2acc0

Browse files
corocoro
authored
Migrating Concourse CI pipelines to GH Actions (#1208)
## Summary Of Changes - Move the cluster operator to be a multiarch OCI built on AMD64 and ARM64, instead of just the former. - Move existing Concourse CI infrastructure to GitHub Actions ## Additional Context The pipeline now runs, on PRs: - kubectl tests - Unit tests - Integration tests - Tests that a multiarch OCI can be successfully built, but does not push it - Runs system tests locally against kind, for multiple RabbitMQ Cluster versions - This excludes the volume expansion system tests - Runs the documented examples against kind in 'dry-run' mode On pushes to main: - kubectl tests - Unit tests - Integration tests - Builds a multiarch OCI of the operator (pushed to `rabbitmqoperator/cluster-operator` with a tag of the SHA256 and `edge`), and a manifest referencing that OCI - Also builds a single-arch version of the image, for AMD64, for compatibility with internal tooling - Runs system tests against GKE, for multiple RabbitMQ Cluster versions - Runs the documented examples against GKE - Tests if upgrading to the operator built earlier would cause a rolling restart of RabbitmqClusters On creation of a tag in the repo: - Same as push to main, except in addition: - OCI is tagged also with `latest` if the latest version, and the semver version of the release - Uploads manifests to a release in GitHub as a Draft release N.B.: This PR removes the use of the rabbitmqoperator/cluster-operator-dev image, as it's now functionally replaced by the regular image with a SHA tag.
1 parent ebf2075 commit 0f2acc0

File tree

15 files changed

+549
-145
lines changed

15 files changed

+549
-145
lines changed

‎.github/workflows/build-test-publish.yml‎

Lines changed: 386 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,386 @@
1+
name: Build, Test, Publish Image & Manifest
2+
3+
on:
4+
push:
5+
branches: [ "main" ]
6+
paths-ignore:
7+
- 'docs/**'
8+
- '*.md'
9+
- 'LICENSE.txt'
10+
- 'PROJECT'
11+
tags: [ "v*" ]
12+
pull_request:
13+
branches: [ "main" ]
14+
15+
env:
16+
GO_VERSION: ~1.19
17+
# Taken from https://github.com/kubernetes-sigs/kind/releases/tag/v0.17.0
18+
# The image here should be listed under 'Images built for this release' for the version of kind in go.mod
19+
KIND_NODE_IMAGE: "kindest/node:v1.25.3@sha256:f52781bc0d7a19fb6c405c2af83abfeb311f130707a0e219175677e366cc45d1"
20+
BASELINE_UPGRADE_VERSION: v1.12.0
21+
22+
jobs:
23+
kubectl_tests:
24+
name: kubectl rabbitmq tests
25+
runs-on: ubuntu-latest
26+
steps:
27+
- name: Install Go
28+
uses: actions/setup-go@v3
29+
with:
30+
go-version: ${{ env.GO_VERSION }}
31+
check-latest: true
32+
- name: Install Bats
33+
run: |
34+
git clone https://github.com/bats-core/bats-core.git "$HOME"/bats-core
35+
cd "$HOME"/bats-core
36+
sudo ./install.sh /usr/local
37+
- name: Check out code into the Go module directory
38+
uses: actions/checkout@v3
39+
- name: kubectl rabbitmq tests
40+
run: |
41+
make install-tools
42+
kind create cluster --image "${{ env.KIND_NODE_IMAGE }}"
43+
DOCKER_REGISTRY_SERVER=local-server OPERATOR_IMAGE=local-operator make deploy-kind kubectl-plugin-tests
44+
unit_integration_tests:
45+
name: unit and integration tests
46+
runs-on: ubuntu-latest
47+
container: us.gcr.io/cf-rabbitmq-for-k8s-bunny/rabbitmq-for-kubernetes-ci
48+
steps:
49+
- name: Install Go
50+
uses: actions/setup-go@v3
51+
with:
52+
go-version: ${{ env.GO_VERSION }}
53+
check-latest: true
54+
- name: Check out code into the Go module directory
55+
uses: actions/checkout@v3
56+
- name: Unit tests
57+
run: make unit-tests
58+
- name: Integration tests
59+
run: make integration-tests
60+
build_operator:
61+
runs-on: ubuntu-latest
62+
needs: unit_integration_tests
63+
permissions:
64+
contents: 'write'
65+
id-token: 'write'
66+
steps:
67+
- name: Checkout
68+
uses: actions/checkout@v3
69+
- name: Install Go
70+
uses: actions/setup-go@v3
71+
with:
72+
go-version: ${{ env.GO_VERSION }}
73+
check-latest: true
74+
- name: OCI Metadata for multi-arch image
75+
id: meta
76+
uses: docker/metadata-action@v4
77+
with:
78+
# list of Docker images to use as base name for tags
79+
images: |
80+
rabbitmqoperator/cluster-operator
81+
quay.io/rabbitmqoperator/cluster-operator
82+
# generate Docker tags based on the following events/attributes
83+
tags: |
84+
type=edge
85+
type=sha
86+
type=ref,event=pr
87+
type=semver,pattern={{version}}
88+
type=semver,pattern={{major}}.{{minor}}
89+
type=semver,pattern={{major}},enable=${{ !startsWith(github.ref, 'refs/tags/v0.') }}
90+
- name: Set up QEMU
91+
uses: docker/setup-qemu-action@v2
92+
- name: Set up Docker Buildx
93+
uses: docker/setup-buildx-action@v2
94+
- name: Login to Docker Hub
95+
if: github.event_name != 'pull_request'
96+
uses: docker/login-action@v2
97+
with:
98+
username: ${{ secrets.DOCKERHUB_USERNAME }}
99+
password: ${{ secrets.DOCKERHUB_TOKEN }}
100+
- name: Login to Quay.io
101+
if: github.event_name != 'pull_request'
102+
uses: docker/login-action@v2
103+
with:
104+
registry: quay.io
105+
username: ${{ secrets.QUAY_USERNAME }}
106+
password: ${{ secrets.QUAY_ROBOT_TOKEN }}
107+
- name: Build and push
108+
uses: docker/build-push-action@v3
109+
with:
110+
context: .
111+
platforms: linux/amd64, linux/arm64
112+
push: ${{ github.event_name != 'pull_request' }}
113+
tags: ${{ steps.meta.outputs.tags }}
114+
labels: ${{ steps.meta.outputs.labels }}
115+
- name: OCI Metadata for single-arch image
116+
if: startsWith(github.ref, 'refs/tags/v')
117+
id: single-arch-meta
118+
uses: docker/metadata-action@v4
119+
with:
120+
# list of Docker images to use as base name for tags
121+
images: |
122+
rabbitmqoperator/cluster-operator
123+
flavor: |
124+
latest=false
125+
# generate Docker tags based on the following events/attributes
126+
tags: |
127+
type=semver,pattern={{version}},suffix=-amd64,latest=false
128+
- name: Build and push single-arch image
129+
if: startsWith(github.ref, 'refs/tags/v')
130+
uses: docker/build-push-action@v3
131+
with:
132+
context: .
133+
platforms: linux/amd64
134+
push: ${{ github.event_name != 'pull_request' }}
135+
tags: ${{ steps.single-arch-meta.outputs.tags }}
136+
labels: ${{ steps.single-arch-meta.outputs.labels }}
137+
- name: Build manifest
138+
if: github.event_name != 'pull_request'
139+
env:
140+
RELEASE_VERSION: ${{ steps.meta.outputs.version }}
141+
run: |
142+
make install-tools
143+
pushd config/installation
144+
kustomize edit set image rabbitmqoperator/cluster-operator-dev=rabbitmqoperator/cluster-operator:"${RELEASE_VERSION}"
145+
popd
146+
make generate-installation-manifest QUAY_IO_OPERATOR_IMAGE=quay.io/rabbitmqoperator/cluster-operator:"${RELEASE_VERSION}"
147+
- name: Upload operator manifests
148+
if: github.event_name != 'pull_request'
149+
uses: actions/upload-artifact@v3
150+
with:
151+
name: operator-manifests
152+
path: releases/cluster-operator*.yml
153+
retention-days: 2
154+
if-no-files-found: error
155+
- name: Rename manifest for GCS
156+
if: github.event_name != 'pull_request'
157+
run: mv releases/cluster-operator.yml cluster-operator-${{ steps.meta.outputs.version }}.yml
158+
- id: 'auth'
159+
uses: 'google-github-actions/auth@v1'
160+
with:
161+
workload_identity_provider: ${{ secrets.GCP_IDENTITY_PROVIDER }}
162+
service_account: ${{ secrets.GCP_SA }}
163+
- name: Upload manifests to GCS
164+
if: github.event_name != 'pull_request'
165+
uses: 'google-github-actions/upload-cloud-storage@v1'
166+
with:
167+
path: cluster-operator-${{ steps.meta.outputs.version }}.yml
168+
destination: operator-manifests-dev
169+
system_tests_local:
170+
name: Local system tests (using KinD)
171+
runs-on: ubuntu-latest
172+
if: ${{ github.event_name == 'pull_request' }}
173+
needs: build_operator
174+
strategy:
175+
matrix:
176+
rabbitmq-image:
177+
- rabbitmq:3.9.9-management
178+
- rabbitmq:management
179+
- pivotalrabbitmq/rabbitmq:main-otp-min-bazel
180+
- pivotalrabbitmq/rabbitmq:main-otp-max-bazel
181+
steps:
182+
- name: Install Go
183+
uses: actions/setup-go@v3
184+
with:
185+
go-version: ${{ env.GO_VERSION }}
186+
check-latest: true
187+
- name: Check out code into the Go module directory
188+
uses: actions/checkout@v3
189+
- name: System tests
190+
env:
191+
KIND_NODE_IMAGE: ${{ env.KIND_NODE_IMAGE }}
192+
RABBITMQ_IMAGE: ${{ matrix.rabbitmq-image }}
193+
run: |
194+
make install-tools
195+
kind create cluster --image "$KIND_NODE_IMAGE"
196+
DOCKER_REGISTRY_SERVER=local-server OPERATOR_IMAGE=local-operator make deploy-kind
197+
make cert-manager
198+
SUPPORT_VOLUME_EXPANSION=false make system-tests
199+
- name: Dry-run example YAMLs
200+
run: |
201+
for a in docs/examples/*/*.y*ml; do
202+
manifest_path="$manifest_path -f $a"
203+
done
204+
# The examples are assumed to run in the 'examples' namespace, create if it doesn't exist
205+
kubectl create namespace examples --dry-run=client -o yaml | kubectl apply -f -
206+
kubectl --namespace=examples apply --dry-run=server $manifest_path
207+
system_tests:
208+
name: System tests
209+
runs-on: ubuntu-latest
210+
container: us.gcr.io/cf-rabbitmq-for-k8s-bunny/rabbitmq-for-kubernetes-ci
211+
if: ${{ github.event_name != 'pull_request' }}
212+
# Add "id-token" with the intended permissions.
213+
permissions:
214+
contents: 'write'
215+
id-token: 'write'
216+
needs: build_operator
217+
strategy:
218+
matrix:
219+
rabbitmq-image:
220+
- rabbitmq:3.9.9-management
221+
- rabbitmq:management
222+
- pivotalrabbitmq/rabbitmq:main-otp-min-bazel
223+
- pivotalrabbitmq/rabbitmq:main-otp-max-bazel
224+
include:
225+
- rabbitmq-image: rabbitmq:3.9.9-management
226+
gke-cluster: ci-bunny-1
227+
- rabbitmq-image: rabbitmq:management
228+
gke-cluster: ci-bunny-1
229+
- rabbitmq-image: pivotalrabbitmq/rabbitmq:main-otp-min-bazel
230+
gke-cluster: ci-bunny-2
231+
- rabbitmq-image: pivotalrabbitmq/rabbitmq:main-otp-max-bazel
232+
gke-cluster: ci-bunny-2
233+
steps:
234+
- name: Install Go
235+
uses: actions/setup-go@v3
236+
with:
237+
go-version: ${{ env.GO_VERSION }}
238+
check-latest: true
239+
- name: Check out code into the Go module directory
240+
uses: actions/checkout@v3
241+
- name: Acquire lock for ${{ matrix.gke-cluster }}
242+
uses: ben-z/gh-action-mutex@9709ba4d8596ad4f9f8bbe8e0f626ae249b1b3ac
243+
with:
244+
branch: lock-${{ matrix.gke-cluster }}
245+
- id: 'auth'
246+
uses: 'google-github-actions/auth@v1'
247+
with:
248+
workload_identity_provider: ${{ secrets.GCP_IDENTITY_PROVIDER }}
249+
service_account: ${{ secrets.GCP_SA }}
250+
- id: 'get-credentials'
251+
uses: 'google-github-actions/get-gke-credentials@v1'
252+
with:
253+
cluster_name: ${{ matrix.gke-cluster }}
254+
location: europe-west1
255+
- name: Get operator manifest
256+
uses: actions/download-artifact@v3
257+
with:
258+
name: operator-manifests
259+
- name: Install Operator build
260+
run: |
261+
make install-tools
262+
make destroy
263+
kubectl apply -f cluster-operator.yml
264+
kubectl --namespace=rabbitmq-system wait --for=condition=Available deployment/rabbitmq-cluster-operator
265+
- name: System tests
266+
env:
267+
RABBITMQ_IMAGE: ${{ matrix.rabbitmq-image }}
268+
run: |
269+
make system-tests
270+
test_doc_examples:
271+
name: Documented example tests
272+
runs-on: ubuntu-latest
273+
container: us.gcr.io/cf-rabbitmq-for-k8s-bunny/rabbitmq-for-kubernetes-ci
274+
if: ${{ github.event_name != 'pull_request' }}
275+
# Add "id-token" with the intended permissions.
276+
permissions:
277+
contents: 'write'
278+
id-token: 'write'
279+
needs: build_operator
280+
steps:
281+
- name: Install Go
282+
uses: actions/setup-go@v3
283+
with:
284+
go-version: ${{ env.GO_VERSION }}
285+
check-latest: true
286+
- name: Check out code into the Go module directory
287+
uses: actions/checkout@v3
288+
- name: Acquire lock for ci-bunny-2
289+
uses: ben-z/gh-action-mutex@9709ba4d8596ad4f9f8bbe8e0f626ae249b1b3ac
290+
with:
291+
branch: lock-ci-bunny-2
292+
- id: 'auth'
293+
uses: 'google-github-actions/auth@v1'
294+
with:
295+
workload_identity_provider: ${{ secrets.GCP_IDENTITY_PROVIDER }}
296+
service_account: ${{ secrets.GCP_SA }}
297+
- id: 'get-credentials'
298+
uses: 'google-github-actions/get-gke-credentials@v1'
299+
with:
300+
cluster_name: ci-bunny-2
301+
location: europe-west1
302+
- name: Get operator manifest
303+
uses: actions/download-artifact@v3
304+
with:
305+
name: operator-manifests
306+
- name: Install Operator build
307+
run: |
308+
make install-tools
309+
make destroy
310+
kubectl apply -f cluster-operator.yml
311+
kubectl --namespace=rabbitmq-system wait --for=condition=Available deployment/rabbitmq-cluster-operator
312+
- name: Documented example tests
313+
run: docs/examples/test.sh
314+
test_upgrade:
315+
name: Test upgrade of the operator
316+
runs-on: ubuntu-latest
317+
container: us.gcr.io/cf-rabbitmq-for-k8s-bunny/rabbitmq-for-kubernetes-ci
318+
if: ${{ github.event_name != 'pull_request' }}
319+
# Add "id-token" with the intended permissions.
320+
permissions:
321+
contents: 'write'
322+
id-token: 'write'
323+
needs: build_operator
324+
steps:
325+
- name: Install Go
326+
uses: actions/setup-go@v3
327+
with:
328+
go-version: ${{ env.GO_VERSION }}
329+
check-latest: true
330+
- name: Check out code into the Go module directory
331+
uses: actions/checkout@v3
332+
- name: Acquire lock for ci-bunny-1
333+
uses: ben-z/gh-action-mutex@9709ba4d8596ad4f9f8bbe8e0f626ae249b1b3ac
334+
with:
335+
branch: lock-ci-bunny-1
336+
- id: 'auth'
337+
uses: 'google-github-actions/auth@v1'
338+
with:
339+
workload_identity_provider: ${{ secrets.GCP_IDENTITY_PROVIDER }}
340+
service_account: ${{ secrets.GCP_SA }}
341+
- id: 'get-credentials'
342+
uses: 'google-github-actions/get-gke-credentials@v1'
343+
with:
344+
cluster_name: ci-bunny-1
345+
location: europe-west1
346+
- name: Get operator manifest
347+
uses: actions/download-artifact@v3
348+
with:
349+
name: operator-manifests
350+
- name: Test upgrade rollout
351+
run: hack/test-upgrade.sh https://github.com/rabbitmq/cluster-operator/releases/download/${{ env.BASELINE_UPGRADE_VERSION }}/cluster-operator.yml cluster-operator.yml release-header.md
352+
- name: Upload release header
353+
if: github.event_name != 'pull_request'
354+
uses: actions/upload-artifact@v3
355+
with:
356+
name: release-header
357+
path: release-header.md
358+
retention-days: 2
359+
if-no-files-found: error
360+
release:
361+
name: Release to GitHub Releases
362+
runs-on: ubuntu-latest
363+
if: startsWith(github.ref, 'refs/tags/v')
364+
needs: [system_tests, test_doc_examples, test_upgrade]
365+
steps:
366+
- name: Checkout
367+
uses: actions/checkout@v2
368+
- name: Get operator manifest
369+
uses: actions/download-artifact@v3
370+
with:
371+
name: operator-manifests
372+
- name: Get release header
373+
uses: actions/download-artifact@v3
374+
with:
375+
name: release-header
376+
- name: Release
377+
uses: softprops/action-gh-release@de2c0eb89ae2a093876385947365aca7b0e5f844
378+
if: startsWith(github.ref, 'refs/tags/')
379+
with:
380+
files: |
381+
cluster-operator.yml
382+
cluster-operator-quay-io.yml
383+
generate_release_notes: true
384+
draft: true
385+
body_path: release-header.md
386+
fail_on_unmatched_files: true

0 commit comments

Comments
 (0)