📅 6/3 @ 1pm PT - How to update lock files silently

[gr2m]

💁🏻 How to update lock files silently
📅 Thursday, June 3, 2021
🕐 1:00pm Pacific Time (in your timezone)
🎙️ no guests
🏷️ Automation

---

Subscribe to this issues to get a notification before the show begins and a summary after the show concludes.

How to update lock files silently

This show is a follow up to #3

I started using @renovateapp for dependency management in most of my open source projects. Greenkeeper is discontinued (RIP), Dependabot is geared towards apps, not libraries, and I'm not aware of any other good alternative for JavaScript projects.

I liked the way Greenkeeper did in-range dependency update checks. It created a branch for the dependency's in-range update, it ran the CI, if there was no error, the branch was deleted. As a maintainer, I didn't get a single notification in the whole process. If the CI failed, Greenkeeper would create an issue to make me aware of the problem. It would also automatically close that issue if the next version update no longer failed.

Renovate does not offer that option, because it requires to keep state for repositories which Renovate doesn't have. Renovate offers the option to do lock file updates, but it cannot merge the changes into the default branch if it is protected, which results in a huge amount notifications across all the repositories, currently it's 100+ each Sunday late afternoon for me.

As an alternative, I'd like to build a GitHub action which can push changes to a protected default branch. It could run on a schedule, checkout the repository, update all dependencies, commit the changes to package-lock.json. Then it would run the tests. If they all succeeded, the change would be pushed to the default branch. If they failed, the changes would be pushed to a branch and a pull request would opened for investigations.

The action could run quite frequently and push changes to the main branch if there are no open pull requests. And with a less frequent interval it would force an update, even if there are open pull requests, to make sure the main repository is up to date. Another trigger could be when a pull request is merged or closed.

Outline

• showcase the problem with committing directly to protected branches as an app
• commit to a protected branch using GitHub Actions
• update package-lock.json and update the default branch using GitHub Actions once per week
• do the update frequently but only if there are no other open pull requests since the last forced update

Preparation

• 30 minute announcement tweet
• start of show tweet
• comment on issue
• recording available tweet

Recording

Shownotes

• I created https://github.com/gr2m/silent-lockfile-update
• I showcased the problem of why dependency update apps such as renovate cannot update dependencies and lockfiles silently in the background if branch protection is enabled, and how to workaround it using a personal access token or a user-to-server token. The workaround only works if the branch protection rules are not enforced for admins, too.
• I created a workflow that updates the lock file daily and pushes the update to a lockfile-update branch. The branch gets created unless it already exists.
• I'll continue the show next week

[oscard0m]

Still waiting for the upload to YouTube, but will already close the issue as the show is done. Please comment here if you have any questions

https://www.youtube.com/watch?v=2UK_gKqVHBU

[oscard0m]

In my ex-company they are running self-hosted reenovatebot and to fix the problem of required reviewer and required wait to check runs pass we built 2 probot apps:

• mrf-reviewer: it auto-approves a PR when the author is renovatebot (we added an extra functionality which was, a PR with title containing keyword [oncall-pr], was usefule for colleagues on-call outside of business hours)
• mrf-merger: listens to check runs success and merges and deletes the pr branch

If you have any questions on that let me know, maybe I can help.

---

Just to understand the purpose of this task:

• You want to get rid of renovate in order to remove noise of pull requests in GitHub's notifications for yo and your contributors?
• And doing so, you plan to assume the cost of pushing directly to defaultBranch upgrade of packages which could cost bugs and regressions?
• Releasing a new package would be avoided by Ci I guess but, you would block contributors of landing other fixes until that PR issue es addressed or last bump of package-lock is reverted, right?

[gr2m]

I want to avoid pull requests altogether. Even when they are auto approved and merged, it still creates tons of notifications. And for something like lock file updates I think the noise is quite distracting.

You want to get rid of renovate in order to remove noise of pull requests in GitHub's notifications for yo and your contributors?

I don't want to get rid of renovate, I just want to get rid of pull requests for lock file updates

And doing so, you plan to assume the cost of pushing directly to defaultBranch upgrade of packages which could cost bugs and regressions?

Sorry I don't understand what you mean? As this is only about lock file updates, there is now way that regressions could be introduced.

Releasing a new package would be avoided by Ci I guess but, you would block contributors of landing other fixes until that PR issue es addressed or last bump of package-lock is reverted, right?

Yes, Ideally I still want to enforce reviews for pull requests

[oscard0m]

I want to avoid pull requests altogether. Even when they are auto approved and merged, it still creates tons of notifications. And for something like lock file updates I think the noise is quite distracting.

Indeed. In my previous company was crazy.

I don't want to get rid of renovate, I just want to get rid of pull requests for lock file updates

Ok, so the DIY solution for updating the package-lock you used in the show was just to simplify the problem but you plan to stay with renovate. Understood 👍🏽

Sorry I don't understand what you mean? As this is only about lock file updates, there is now way that regressions could be introduced.

If package-lock.json is updated npm ci will get those changes in consideration right? Sounds like I'm missing something about current flow here 😕

Yes, Ideally I still want to enforce reviews for pull requests

I think I'm not following you here. What I meant is:

1. Renovate updates package-lock.json silently into defaultBranch. Since now these kind of updates are merged directly, there won't be any CI check which prevent doing this.
2. After that commit is added to defaultBranch, a workflow runs to check if that update makes fail tests (it runs npm ci to get latest changes in package-lock and npm test)
3. If it fails, it creates an issue to fix it
4. Other existing PR's of that repo become blocked until that fix is done, because they require to get aligned with latest changes of defaultBranch and that branch is making tests fail.

Is this the flow or am I missing something here? 😕

[gr2m]

Sorry I don't understand what you mean? As this is only about lock file updates, there is now way that regressions could be introduced.

If package-lock.json is updated npm ci will get those changes in consideration right? Sounds like I'm missing something about current flow here 😕

package-lock.json files are not published to npm, its effects are limited to the source repository

1. Renovate updates package-lock.json silently into defaultBranch. Since now these kind of updates are merged directly, there won't be any CI check which prevent doing this.
2. After that commit is added to defaultBranch, a workflow runs to check if that update makes fail tests (it runs npm ci to get latest changes in package-lock and npm test)

There will be CI checks, I just didn't work on that yet. There are no reviews, but there don't need to be any, but I still want to trigger the test runs before anything is merged into the repository's default branch. The changes will happen in a branch such as lockfile-update, and if any tests fail, a pull request is created for the lockfile-update branch to update the maintainers