Skip to content

Commit 84e4901

Browse files
Logiraptordimitarvdimitrovkrajorama
authored
Add a helm golden record (#2034)
* Helm: Add golden-record build script * Helm: add test-values golden record * Add PR check for `check-helm-tests` * Add Helm setup to lint-helm action * Update generated helm tests * Fix bash linting * Update contribution guidelines * Update generated helm manifests * Helm: fix kube version Set kubeVersionOverride to generate PodDisruptionBudget API version consistently. When I ran the test, I got a diff, because my k8s is newer (1.23). Signed-off-by: György Krajcsovits <[email protected]> * Update operations/helm/tests/build.sh Co-authored-by: Dimitar Dimitrov <[email protected]> Co-authored-by: György Krajcsovits <[email protected]>
1 parent 8415eb1 commit 84e4901

File tree

101 files changed

+4627
-0
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

101 files changed

+4627
-0
lines changed

‎.github/workflows/test-build-deploy.yml‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -71,6 +71,26 @@ jobs:
7171
- name: Check Jsonnet Tests
7272
run: make BUILD_IN_CONTAINER=false check-jsonnet-tests
7373

74+
lint-helm:
75+
runs-on: ubuntu-latest
76+
container:
77+
image: grafana/mimir-build-image:publish-multiarch-images-7a4b40a6d
78+
steps:
79+
- name: Checkout Repo
80+
uses: actions/checkout@v2
81+
# Commands in the Makefile are hardcoded with an assumed file structure of the CI container
82+
# Symlink ensures paths specified in previous commands don’t break
83+
- name: Symlink Expected Path to Workspace
84+
run: |
85+
mkdir -p /go/src/github.com/grafana/mimir
86+
ln -s $GITHUB_WORKSPACE/* /go/src/github.com/grafana/mimir
87+
- name: Set up Helm
88+
uses: azure/setup-helm@v1
89+
with:
90+
version: v3.5.2
91+
- name: Check Helm Tests
92+
run: make BUILD_IN_CONTAINER=false check-helm-tests
93+
7494
test:
7595
runs-on: ubuntu-latest
7696
strategy:

‎Makefile‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -496,6 +496,12 @@ check-jsonnet-getting-started:
496496
| sed 's/\(jb install github.com\/grafana\/mimir\/operations\/mimir@main\)/\1 \&\& rm -fr .\/vendor\/mimir \&\& cp -r ..\/operations\/mimir .\/vendor\/mimir\//g' \
497497
| bash
498498

499+
build-helm-tests:
500+
@./operations/helm/tests/build.sh
501+
502+
check-helm-tests: build-helm-tests
503+
@git diff --exit-code -- ./operations/helm/tests || (echo "Please rebuild helm tests output 'make build-helm-tests'" && false)
504+
499505
build-jsonnet-tests:
500506
@./operations/mimir-tests/build.sh
501507

‎docs/internal/contributing/contributing-to-helm-chart.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,13 @@ Please see the [general workflow](README.md#workflow) for reference.
66

77
- Changelog is in the chart itself [operations/helm/charts/mimir-distributed/CHANGELOG.md](https://github.com/grafana/mimir/blob/main/operations/helm/charts/mimir-distributed/CHANGELOG.md).
88
- If you made any changes to the [operations/helm/charts/mimir-distributed/Chart.yaml](https://github.com/grafana/mimir/blob/main/operations/helm/charts/mimir-distributed/Chart.yaml), run `make doc` and commit the changed files to update the [operations/helm/charts/mimir-distributed/README.md](https://github.com/grafana/mimir/blob/main/operations/helm/charts/mimir-distributed/README.md).
9+
- If your changes impact the test configurations in the [operations/helm/charts/mimir-distributed/ci](https://github.com/grafana/mimir/blob/main/operations/helm/charts/mimir-distributed/ci) directory, see [Updating compiled manifests](#updating-compiled-manifests).
10+
11+
## Updating compiled manifests
12+
13+
We keep a compiled version of the helm chart for each values file in the `ci` directory.
14+
This makes it easy to see how a given PR impacts the final output.
15+
A PR check will fail if you forget to update the compiled manifests, and you can use `make build-helm-tests` to update them.
916

1017
## Versioning
1118

‎operations/helm/charts/mimir-distributed/ci/test-enterprise-values.yaml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,6 @@
11
# Test values to limit the load during CI
2+
kubeVersionOverride: "1.20"
3+
24
enterprise:
35
enabled: true
46

‎operations/helm/charts/mimir-distributed/ci/test-oss-values.yaml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,6 @@
11
# Test values to limit the load during CI
2+
kubeVersionOverride: "1.20"
3+
24
alertmanager:
35
persistentVolume:
46
enabled: false

‎operations/helm/tests/build.sh‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
#!/usr/bin/env bash
2+
# SPDX-License-Identifier: AGPL-3.0-only
3+
4+
set -euo pipefail
5+
6+
CHART_PATH=operations/helm/charts/mimir-distributed
7+
8+
# Start from a clean slate
9+
rm -rf operations/helm/tests/*-generated
10+
11+
# Install chart dependencies for this branch
12+
helm dependency update "$CHART_PATH"
13+
14+
# Locally render the chart for every test file
15+
TESTS=$(ls -1 ${CHART_PATH}/ci/*values.yaml)
16+
17+
for FILEPATH in $TESTS; do
18+
# Extract the filename (without extension).
19+
TEST_NAME=$(basename -s '.yaml' "$FILEPATH")
20+
21+
echo "Templating $TEST_NAME"
22+
helm template "${TEST_NAME}" ${CHART_PATH} -f "${FILEPATH}" --output-dir "operations/helm/tests/${TEST_NAME}-generated"
23+
done

‎operations/helm/tests/test-enterprise-values-generated/mimir-distributed/charts/minio/templates/configmap.yaml‎

Lines changed: 111 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,111 @@
1+
---
2+
# Source: mimir-distributed/charts/minio/templates/configmap.yaml
3+
apiVersion: v1
4+
kind: ConfigMap
5+
metadata:
6+
name: test-enterprise-values-minio
7+
labels:
8+
app: minio
9+
chart: minio-8.0.10
10+
release: test-enterprise-values
11+
heritage: Helm
12+
data:
13+
initialize: |-
14+
#!/bin/sh
15+
set -e ; # Have script exit in the event of a failed command.
16+
MC_CONFIG_DIR="/etc/minio/mc/"
17+
MC="/usr/bin/mc --insecure --config-dir ${MC_CONFIG_DIR}"
18+
19+
# connectToMinio
20+
# Use a check-sleep-check loop to wait for Minio service to be available
21+
connectToMinio() {
22+
SCHEME=$1
23+
ATTEMPTS=0 ; LIMIT=29 ; # Allow 30 attempts
24+
set -e ; # fail if we can't read the keys.
25+
ACCESS=$(cat /config/accesskey) ; SECRET=$(cat /config/secretkey) ;
26+
set +e ; # The connections to minio are allowed to fail.
27+
echo "Connecting to Minio server: $SCHEME://$MINIO_ENDPOINT:$MINIO_PORT" ;
28+
MC_COMMAND="${MC} config host add myminio $SCHEME://$MINIO_ENDPOINT:$MINIO_PORT $ACCESS $SECRET" ;
29+
$MC_COMMAND ;
30+
STATUS=$? ;
31+
until [ $STATUS = 0 ]
32+
do
33+
ATTEMPTS=`expr $ATTEMPTS + 1` ;
34+
echo \"Failed attempts: $ATTEMPTS\" ;
35+
if [ $ATTEMPTS -gt $LIMIT ]; then
36+
exit 1 ;
37+
fi ;
38+
sleep 2 ; # 1 second intervals between attempts
39+
$MC_COMMAND ;
40+
STATUS=$? ;
41+
done ;
42+
set -e ; # reset `e` as active
43+
return 0
44+
}
45+
46+
# checkBucketExists ($bucket)
47+
# Check if the bucket exists, by using the exit code of `mc ls`
48+
checkBucketExists() {
49+
BUCKET=$1
50+
CMD=$(${MC} ls myminio/$BUCKET > /dev/null 2>&1)
51+
return $?
52+
}
53+
54+
# createBucket ($bucket, $policy, $purge)
55+
# Ensure bucket exists, purging if asked to
56+
createBucket() {
57+
BUCKET=$1
58+
POLICY=$2
59+
PURGE=$3
60+
VERSIONING=$4
61+
62+
# Purge the bucket, if set & exists
63+
# Since PURGE is user input, check explicitly for `true`
64+
if [ $PURGE = true ]; then
65+
if checkBucketExists $BUCKET ; then
66+
echo "Purging bucket '$BUCKET'."
67+
set +e ; # don't exit if this fails
68+
${MC} rm -r --force myminio/$BUCKET
69+
set -e ; # reset `e` as active
70+
else
71+
echo "Bucket '$BUCKET' does not exist, skipping purge."
72+
fi
73+
fi
74+
75+
# Create the bucket if it does not exist
76+
if ! checkBucketExists $BUCKET ; then
77+
echo "Creating bucket '$BUCKET'"
78+
${MC} mb myminio/$BUCKET
79+
else
80+
echo "Bucket '$BUCKET' already exists."
81+
fi
82+
83+
84+
# set versioning for bucket
85+
if [ ! -z $VERSIONING ] ; then
86+
if [ $VERSIONING = true ] ; then
87+
echo "Enabling versioning for '$BUCKET'"
88+
${MC} version enable myminio/$BUCKET
89+
elif [ $VERSIONING = false ] ; then
90+
echo "Suspending versioning for '$BUCKET'"
91+
${MC} version suspend myminio/$BUCKET
92+
fi
93+
else
94+
echo "Bucket '$BUCKET' versioning unchanged."
95+
fi
96+
97+
# At this point, the bucket should exist, skip checking for existence
98+
# Set policy on the bucket
99+
echo "Setting policy of bucket '$BUCKET' to '$POLICY'."
100+
${MC} policy set $POLICY myminio/$BUCKET
101+
}
102+
103+
# Try connecting to Minio instance
104+
scheme=http
105+
connectToMinio $scheme
106+
# Create the buckets
107+
createBucket mimir-tsdb none false
108+
createBucket mimir-ruler none false
109+
createBucket enterprise-metrics-tsdb none false
110+
createBucket enterprise-metrics-admin none false
111+
createBucket enterprise-metrics-ruler none false

‎operations/helm/tests/test-enterprise-values-generated/mimir-distributed/charts/minio/templates/deployment.yaml‎

Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
---
2+
# Source: mimir-distributed/charts/minio/templates/deployment.yaml
3+
apiVersion: apps/v1
4+
kind: Deployment
5+
metadata:
6+
name: test-enterprise-values-minio
7+
labels:
8+
app: minio
9+
chart: minio-8.0.10
10+
release: test-enterprise-values
11+
heritage: Helm
12+
spec:
13+
strategy:
14+
type: RollingUpdate
15+
rollingUpdate:
16+
maxSurge: 100%
17+
maxUnavailable: 0
18+
selector:
19+
matchLabels:
20+
app: minio
21+
release: test-enterprise-values
22+
template:
23+
metadata:
24+
name: test-enterprise-values-minio
25+
labels:
26+
app: minio
27+
release: test-enterprise-values
28+
annotations:
29+
checksum/secrets: aac2dea246043210c2649ffaf6c6ea57cd94d0aaf2b21759b8b38a093096478e
30+
checksum/config: 8109517e3e9f729fb84cca8217b59099354497df0ca310f4e4b07d4951b02dc4
31+
spec:
32+
serviceAccountName: "test-enterprise-values-minio"
33+
securityContext:
34+
runAsUser: 1000
35+
runAsGroup: 1000
36+
fsGroup: 1000
37+
containers:
38+
- name: minio
39+
image: "minio/minio:RELEASE.2021-02-14T04-01-33Z"
40+
imagePullPolicy: IfNotPresent
41+
command:
42+
- "/bin/sh"
43+
- "-ce"
44+
- "/usr/bin/docker-entrypoint.sh minio -S /etc/minio/certs/ server /export"
45+
volumeMounts:
46+
- name: export
47+
mountPath: /export
48+
ports:
49+
- name: http
50+
containerPort: 9000
51+
env:
52+
- name: MINIO_ACCESS_KEY
53+
valueFrom:
54+
secretKeyRef:
55+
name: test-enterprise-values-minio
56+
key: accesskey
57+
- name: MINIO_SECRET_KEY
58+
valueFrom:
59+
secretKeyRef:
60+
name: test-enterprise-values-minio
61+
key: secretkey
62+
resources:
63+
requests:
64+
cpu: 100m
65+
memory: 128Mi
66+
volumes:
67+
- name: export
68+
persistentVolumeClaim:
69+
claimName: test-enterprise-values-minio
70+
- name: minio-user
71+
secret:
72+
secretName: test-enterprise-values-minio

‎operations/helm/tests/test-enterprise-values-generated/mimir-distributed/charts/minio/templates/post-install-create-bucket-job.yaml‎

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
---
2+
# Source: mimir-distributed/charts/minio/templates/post-install-create-bucket-job.yaml
3+
apiVersion: batch/v1
4+
kind: Job
5+
metadata:
6+
name: test-enterprise-values-minio-make-bucket-job
7+
labels:
8+
app: minio-make-bucket-job
9+
chart: minio-8.0.10
10+
release: test-enterprise-values
11+
heritage: Helm
12+
annotations:
13+
"helm.sh/hook": post-install,post-upgrade
14+
"helm.sh/hook-delete-policy": hook-succeeded,before-hook-creation
15+
spec:
16+
template:
17+
metadata:
18+
labels:
19+
app: minio-job
20+
release: test-enterprise-values
21+
spec:
22+
restartPolicy: OnFailure
23+
volumes:
24+
- name: minio-configuration
25+
projected:
26+
sources:
27+
- configMap:
28+
name: test-enterprise-values-minio
29+
- secret:
30+
name: test-enterprise-values-minio
31+
serviceAccountName: "test-enterprise-values-minio"
32+
containers:
33+
- name: minio-mc
34+
image: "minio/mc:RELEASE.2021-02-14T04-28-06Z"
35+
imagePullPolicy: IfNotPresent
36+
command: ["/bin/sh", "/config/initialize"]
37+
env:
38+
- name: MINIO_ENDPOINT
39+
value: test-enterprise-values-minio
40+
- name: MINIO_PORT
41+
value: "9000"
42+
volumeMounts:
43+
- name: minio-configuration
44+
mountPath: /config
45+
resources:
46+
requests:
47+
memory: 128Mi

‎operations/helm/tests/test-enterprise-values-generated/mimir-distributed/charts/minio/templates/post-install-prometheus-metrics-role.yaml‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
---
2+
# Source: mimir-distributed/charts/minio/templates/post-install-prometheus-metrics-role.yaml
3+
apiVersion: rbac.authorization.k8s.io/v1
4+
kind: Role
5+
metadata:
6+
name: test-enterprise-values-minio-update-prometheus-secret
7+
labels:
8+
app: minio-update-prometheus-secret
9+
chart: minio-8.0.10
10+
release: test-enterprise-values
11+
heritage: Helm
12+
rules:
13+
- apiGroups:
14+
- ""
15+
resources:
16+
- secrets
17+
verbs:
18+
- get
19+
- create
20+
- update
21+
- patch
22+
resourceNames:
23+
- test-enterprise-values-minio-prometheus
24+
- apiGroups:
25+
- ""
26+
resources:
27+
- secrets
28+
verbs:
29+
- create
30+
- apiGroups:
31+
- monitoring.coreos.com
32+
resources:
33+
- servicemonitors
34+
verbs:
35+
- get
36+
resourceNames:
37+
- test-enterprise-values-minio

0 commit comments

Comments
 (0)