feat: Added Renovate to auto update helm-values and github-actions #979

kvanzuijlen · 2024-01-03T16:04:24Z

TODO:

Write breaking changes docs
Wait for feat(manager/helm-values): Add support for bumpVersion renovatebot/renovate#26441 to be merged
Verify everything is working
Add more renovate post update actions; write changelog etc.

What does this PR do?

This PR adds the "self-hosted" Renovate Github Action.

Fixes Switch from dependabot to renovate #975

If you modified files in the ./charts/jenkins/ directory, please also include the following:

Submitter checklist

Give feedback

I bumped the "version" key in ./charts/jenkins/Chart.yaml.
I added a new changelog entry to ./charts/jenkins/CHANGELOG.md.
I followed the technical requirements.
Options

Special notes for your reviewer

I contributed to Renovate to support bumping the chart version with updates to a Helm values file. We have to wait until that gets merged. I also added support for upgrading the versions for tools that are installed within github-actions. If all of this works the only dependency changes that have to be made manually is the Kind version matrix. I might also be able to take a look at those as well in a future PR.

kvanzuijlen · 2024-01-03T16:19:21Z

@timja What do you think of this direction? I had to make some major (breaking) changes for Renovate to detect updates correctly (https://docs.renovatebot.com/modules/manager/helm-values/#additional-information). This would of course be a major version bump. If you would like to include some other breaking changes let me know!

Signed-off-by: kvanzuijlen <[email protected]>

# Conflicts: # charts/jenkins/unittests/jcasc-config-test.yaml

timja

general approach looks good, i've left a couple of comments.

.github/workflows/renovate.yaml

timja · 2024-01-03T22:35:59Z

.github/renovate.json5

+      matchManagers: ["helm-values"],
+      bumpVersion: "minor",
+      postUpgradeTasks: {
+        "commands": ["helm unittest --strict -f 'unittests/*.yaml' charts/jenkins -u"],


is this needed? won't CI just run this anyway?

The -u updates the snapshots, which is needed for kiwigrid/k8s-sidecar and jenkins/inbound-agent

ah that's cool, much simpler.

We could also configure this to update plugins too right then?

helm-charts/charts/jenkins/values.yaml

Lines 250 to 253 in 762105a

- kubernetes:4029.v5712230ccb_f8

- workflow-aggregator:596.v8c21c963d92d

- git:5.1.0

- configuration-as-code:1670.v564dc8b_982d0

Hmm, that could be a bit harder... We'd need to use the regex manager for that, so we can't use the bumpVersion functionality. I see 2 solutions for that;

Use postUpgradeTasks for bumping the Chart version as well

Use a 2-step approach, where we'd first create a draft PR using renovate and then use another GitHub Action to do stuff like bumping the Chart version, adding to the changelog, etc.

I need to verify if the postUpgradeTasks approach would work, as I don't know if things like helm (and helm unittest) are available once installed on the runner or if we'd need to build a custom Renovate image (Renovate runs in a container, but could, for example, mount some stuff).

If a custom image is required we could opt for option 2 instead. If a custom image isn't a big deal (it should be quite easy, with Renovate auto-updating it, etc.) that would also be an option.

maybe we could maintain a plugins.txt file and sync it in the renovate workflow using yq to override what's in the values.yaml file in post upgrade tasks?

https://docs.renovatebot.com/modules/manager/jenkins/

That would be an option yeah. That still wouldn't solve the bumpVersion issue though. Also, postUpgradeTasks run within the image, so we'd have to install the tools on the runner and mount the executables/binaries, use a custom image, or use the 2-step solution I proposed.

Personally, I'm more of a fan of mounting the tools since it doesn't involve custom images (maintenance!) and it doesn't require a 2-step approach.

kvanzuijlen · 2024-01-18T19:22:58Z

Hi,
Just adding a comment to show I didn't forget about this. My PR for Renovate got merged and released so I'll finish this PR up.

For kindest/node I'm working on a CLI tool that would automate selecting the most recent n versions. Let me know if that's something you'd be interested in.

…te-autoupdate # Conflicts: # charts/jenkins/unittests/jcasc-config-test.yaml

# Conflicts: # charts/jenkins/unittests/jcasc-config-test.yaml # charts/jenkins/unittests/jenkins-controller-statefulset-test.yaml

timja · 2024-01-18T20:02:20Z

For kindest/node I'm working on a CLI tool that would automate selecting the most recent n versions. Let me know if that's something you'd be interested in.

Yes definitely

kvanzuijlen · 2024-01-18T22:28:04Z

@timja I'm planning on testing this by making a copy of my fork (import the fork as a new repository).

I'll merge this branch on that repository and make sure I have the necessary (dummy) secrets set.
I'll then change some of the versions to older versions (where needed) to see if the whole process works as expected. I'll share the public URL once testing is complete.

kvanzuijlen · 2024-01-20T23:06:24Z

@timja as I expected, I needed to make (a ton of) changes to get it to work.

I'm getting closer and closer to getting it done. You can take a look at my progress here: https://github.com/kvanzuijlen/jenkins-helm-charts-renovate

Once I've verified everything is working I'll update this PR and mark it as ready for review. Of course, I want to deliver something workable/maintainable, so please feel free to request any changes/clarifications where needed when the PR is ready for review.

kvanzuijlen · 2024-01-30T21:08:01Z

@timja still haven't forgotten about this. As far as I can tell the following we'd like to automate the following:

Automatically add CHANGELOG entry/bump Chart version/artifacthub.io/images annotation/update tests
- Automatically update plugins (https://github.com/kvanzuijlen/jenkins-helm-charts-renovate/pull/54)
- Automatically update Jenkins (https://github.com/kvanzuijlen/jenkins-helm-charts-renovate/pull/49)
- Automatically update inbound-agent (https://github.com/kvanzuijlen/jenkins-helm-charts-renovate/pull/52)
- Automatically update other images in values.yaml (https://github.com/kvanzuijlen/jenkins-helm-charts-renovate/pull/50)
Automatically update GitHub Actions (https://github.com/kvanzuijlen/jenkins-helm-charts-renovate/pull/53)
Automatically update versions used in GitHub Actions (https://github.com/kvanzuijlen/jenkins-helm-charts-renovate/pull/24)

Am I missing anything here? Can you check the PRs I've added as examples to see if there's anything weird/wrong?

If everything is as desired I'll be resolving the conflicts/updating the configuration. Right now I've left VALUES_SUMMARY.md out-of-scope, as I'd like to propose to start using https://github.com/norwoodj/helm-docs instead. Similar to how https://github.com/renovatebot/helm-charts does it.

timja · 2024-01-30T22:40:38Z

That all looks great thanks for all this, let me know when it's ready for a review!

# Conflicts: # .github/workflows/sync-lts.yaml # charts/jenkins/CHANGELOG.md # charts/jenkins/Chart.yaml # charts/jenkins/unittests/jenkins-controller-statefulset-test.yaml # charts/jenkins/values.yaml

kvanzuijlen

@timja should I also remove the backup section in this major version?

kvanzuijlen · 2024-01-31T09:29:01Z

.github/renovate-config.json5

+  platform: "github",
+  repositories: ["jenkinsci/helm-charts"],
+  allowedPostUpgradeCommands: ["^\.github\/renovate-postupgrade\.sh {{{depName}}} {{{newVersion}}}$"],
+  prConcurrentLimit: 0,


Default is 10, 0 is unlimited. Hourly limit is 2 by default.

kvanzuijlen · 2024-01-31T09:29:54Z

.github/workflows/sync-lts.yaml

Replaced by Renovate

kvanzuijlen · 2024-01-31T09:30:29Z

charts/jenkins/CHANGELOG.md

@@ -12,11 +12,39 @@ Use the following links to reference issues, PRs, and commits prior to v2.6.0.
 The changelog until v1.5.7 was auto-generated based on git commits.
 Those entries include a reference to the git commit to be able to get more details.

+## 5.0.0


Need to include the breaking changes to adminSecrets at the time of writing this comment.

kvanzuijlen · 2024-01-31T09:58:45Z

@timja PR should be ready to review now. Please let me know if anything is unclear/wrong. I'm happy to make improvements where possible/add some explanation if needed.

timja · 2024-01-31T21:59:47Z

@timja should I also remove the backup section in this major version?

yes please

timja

Looks really great, thanks for all the amazing work here, a few nits / questions + there's the backup change for 5.0.0 as well but we'll be able to get this in very soon!

.github/workflows/renovate.yaml

charts/jenkins/CHANGELOG.md

charts/jenkins/README.md

timja

Thanks!

kvanzuijlen force-pushed the renovate-autoupdate branch 3 times, most recently from 9f1c56a to c1b3fa0 Compare January 3, 2024 16:20

kvanzuijlen added 5 commits January 3, 2024 17:23

enhancement: Moved image properties

0741d38

Signed-off-by: kvanzuijlen <[email protected]>

fix(tests): Fixed tests and use snapshots

3a1c0bf

Signed-off-by: kvanzuijlen <[email protected]>

fix(tests): Some refactoring

bdb8786

Signed-off-by: kvanzuijlen <[email protected]>

feat: Added renovate

e517518

Signed-off-by: kvanzuijlen <[email protected]>

chore: cleanup

390836a

Signed-off-by: kvanzuijlen <[email protected]>

kvanzuijlen force-pushed the renovate-autoupdate branch from c1b3fa0 to 390836a Compare January 3, 2024 16:25

kvanzuijlen added 2 commits January 3, 2024 18:08

Merge branch 'main' into renovate-autoupdate

945cafb

# Conflicts: # charts/jenkins/unittests/jcasc-config-test.yaml

Merge branch 'main' into renovate-autoupdate

78cd773

timja reviewed Jan 3, 2024

View reviewed changes

timja mentioned this pull request Jan 17, 2024

Add support for generic ephemeral storage #982

Closed

kvanzuijlen added 2 commits January 18, 2024 20:25

Merge remote-tracking branch 'origin/renovate-autoupdate' into renova…

94553c4

…te-autoupdate # Conflicts: # charts/jenkins/unittests/jcasc-config-test.yaml

Merge branch 'main' into renovate-autoupdate

bf306d6

# Conflicts: # charts/jenkins/unittests/jcasc-config-test.yaml # charts/jenkins/unittests/jenkins-controller-statefulset-test.yaml

kvanzuijlen added 10 commits January 18, 2024 21:04

chore: Updated renovate config and action

42898ff

fix: Fixed mount volumes

2efe39e

docs: Added breaking changes documentation

b14ce8b

fix: Updated test snapshots

71c219a

chore: Also update jenkins version via Renovate

1833cba

chore: Temporarily enable dependency dashboard

12d7783

chore: Fixed md indentation

dba6f97

chore: Made unittests more robust

66626c5

chore: Bump Chart version

980e329

fix: remove erroneous condition

ed53918

kvanzuijlen mentioned this pull request Jan 30, 2024

Minio S3 backup doesn't work due to certificate signed by unknown authority #978

Closed

kvanzuijlen mentioned this pull request Jan 30, 2024

Use https://github.com/norwoodj/helm-docs to generate VALUES_SUMMARY.md #986

Closed

kvanzuijlen added 2 commits January 31, 2024 09:56

Merge branch 'main' into renovate-autoupdate

fb9052c

# Conflicts: # .github/workflows/sync-lts.yaml # charts/jenkins/CHANGELOG.md # charts/jenkins/Chart.yaml # charts/jenkins/unittests/jenkins-controller-statefulset-test.yaml # charts/jenkins/values.yaml

feat: Added Renovate configuration

fc6d514

kvanzuijlen marked this pull request as ready for review January 31, 2024 09:28

kvanzuijlen requested review from a team as code owners January 31, 2024 09:28

kvanzuijlen commented Jan 31, 2024

View reviewed changes

kvanzuijlen added 3 commits January 31, 2024 10:48

chore: Renamed admin credential values

9780bbc

chore: Updated changelog

05e5586

chore: Fixed linting issues

d0b3aad

timja approved these changes Jan 31, 2024

View reviewed changes

.github/workflows/renovate.yaml Show resolved Hide resolved

charts/jenkins/CHANGELOG.md Outdated Show resolved Hide resolved

charts/jenkins/README.md Outdated Show resolved Hide resolved

kvanzuijlen added 5 commits February 1, 2024 10:47

chore: Removed backup from Helm Chart

6e26829

chore: Added to UPGRADING.md and updated deprecation.yaml

49e3943

chore: Fixed linting issues

cc69d9b

chore: Fixed deprecation.yaml

dcb318c

chore: Fixed linting issues

fc65c38

timja approved these changes Feb 1, 2024

View reviewed changes

timja merged commit e2c1450 into jenkinsci:main Feb 1, 2024
5 checks passed

kvanzuijlen mentioned this pull request Feb 2, 2024

Update helm command #800

Closed

3 tasks

kvanzuijlen mentioned this pull request Mar 11, 2024

Automate dependency updates with Renovate grafana/helm-charts#3012

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: Added Renovate to auto update helm-values and github-actions #979

feat: Added Renovate to auto update helm-values and github-actions #979

kvanzuijlen commented Jan 3, 2024 •

edited

Loading

Submitter checklist

kvanzuijlen commented Jan 3, 2024

timja left a comment

timja Jan 3, 2024

kvanzuijlen Jan 3, 2024

timja Jan 4, 2024

kvanzuijlen Jan 4, 2024

timja Jan 4, 2024

kvanzuijlen Jan 4, 2024 •

edited

Loading

kvanzuijlen commented Jan 18, 2024

timja commented Jan 18, 2024

kvanzuijlen commented Jan 18, 2024 •

edited

Loading

kvanzuijlen commented Jan 20, 2024

kvanzuijlen commented Jan 30, 2024

timja commented Jan 30, 2024

kvanzuijlen left a comment

kvanzuijlen Jan 31, 2024

kvanzuijlen Jan 31, 2024

kvanzuijlen Jan 31, 2024

kvanzuijlen commented Jan 31, 2024

timja commented Jan 31, 2024

timja left a comment

timja left a comment

	- kubernetes:4029.v5712230ccb_f8
	- workflow-aggregator:596.v8c21c963d92d
	- git:5.1.0
	- configuration-as-code:1670.v564dc8b_982d0

feat: Added Renovate to auto update helm-values and github-actions #979

feat: Added Renovate to auto update helm-values and github-actions #979

Conversation

kvanzuijlen commented Jan 3, 2024 • edited Loading

What does this PR do?

Submitter checklist

Special notes for your reviewer

kvanzuijlen commented Jan 3, 2024

timja left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kvanzuijlen Jan 4, 2024 • edited Loading

Choose a reason for hiding this comment

kvanzuijlen commented Jan 18, 2024

timja commented Jan 18, 2024

kvanzuijlen commented Jan 18, 2024 • edited Loading

kvanzuijlen commented Jan 20, 2024

kvanzuijlen commented Jan 30, 2024

timja commented Jan 30, 2024

kvanzuijlen left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kvanzuijlen commented Jan 31, 2024

timja commented Jan 31, 2024

timja left a comment

Choose a reason for hiding this comment

timja left a comment

Choose a reason for hiding this comment

kvanzuijlen commented Jan 3, 2024 •

edited

Loading

kvanzuijlen Jan 4, 2024 •

edited

Loading

kvanzuijlen commented Jan 18, 2024 •

edited

Loading