Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conditionally publish deb packages #9496

Merged
merged 9 commits into from
Jan 14, 2022
Merged

Conditionally publish deb packages #9496

merged 9 commits into from
Jan 14, 2022

Conversation

wadells
Copy link
Contributor

@wadells wadells commented Dec 20, 2021

Summary

This is the last interim fix for our deb repos. Specifically, I make the following changes:

  1. deb archives are not published to apt if they're not the latest release ever. This prevents the type of carnage described in Deb repo inconsistent versions available #8166.
  2. Both rpm and deb archives are no longer published to yum / apt if they contain any pre-release indicator or build metadata. This catches stuff like v6.2.14-debug.4 that's slipped into the repos before.
  3. Nothing is published if the commit isn't tagged. This was already true due to the automation's use of DRONE_TAG, but now the failure is quicker, explicit, and has a hopefully helpful error message.

Once this merged to master and 8.0, we won't have to worry about 8.0+ causing https://deb.releases.teleport.dev/ stable main any future releases (e.g. 8.1 or 9.0).

Needs backporting to v8.0, but no further because we have a simpler strategy in 7.0 (#9237), 6.2 (#9238), and 5.0 (#9239). 4.4 and prior predate apt (and are very out of support).

Contributes to #8166.

Testing Done

Updated testing:

walt@work:~/git/teleport/build.assets/version-check$ go run main.go -tag v9.0.0-beta.1 -check latest
walt@work:~/git/teleport/build.assets/version-check$ go run main.go -tag v9.0.0-beta.1 -check prerelease
2022/01/13 13:26:51 Check failed: version is pre-release: v9.0.0-beta.1.
exit status 1
walt@work:~/git/teleport/build.assets/version-check$ go run main.go -tag v8.0.0-beta.1 -check latest
2022/01/13 13:27:07 Check failed: found newer version of release, not releasing. Latest release: v8.1.0, tag: v8.0.0-beta.1.
exit status 1

Because the changed pipeline actually publishes stuff, I wasn't able to test the code in-situ. Instead, I wrote a minimal test harness (see this change), and tried with the following tags: v0.0.0, v8.8.8, v9.0.0.-dev.1. You can find the builds here:

After completing the testing, I cut and paste the steps into the production pipeline, and updated comments. I also deleted all the bogus tags I used for testing.

Notes

Many thanks to @russjones for prototyping the version checker in #9236

I considered that we may see ip based rate limiting / throttling from GitHub, as we don't use an authenticate API client. However, I think this is unlikely because release jobs are relatively infrequent, typically zero up to 3 in a day (if they need to be retried for some reason).

.drone.yml Outdated Show resolved Hide resolved
.drone.yml Show resolved Hide resolved
build.assets/version-check-latest/main.go Outdated Show resolved Hide resolved
build.assets/version-check-prerelease/main.go Outdated Show resolved Hide resolved
build.assets/version-check-prerelease/main.go Outdated Show resolved Hide resolved
@wadells wadells force-pushed the walt/deb branch 2 times, most recently from 656f08f to a9c1337 Compare January 7, 2022 02:31
russjones and others added 3 commits January 11, 2022 19:00
This script can be used to determine if a tag is the highest semver
version of Teleport ever released.  This is useful for conditional build
steps that should only run for the most current release (e.g. updating
'latest' references).

(cherry picked from commit 7390509)
We've seen two debug / prerelease builds slip through to public package
repositories.  This script should help prevent that from happening
again.
This patch makes a couple changes:

  1. deb archives are not published to apt if they're not the latest
     release ever
  2. both rpm and deb archives are no longer published to yum / apt if
     they contain any pre-release indicator or build metadata
  3. nothing is published if the commit isn't tagged.

Contributes to #8166
@wadells
Copy link
Contributor Author

wadells commented Jan 12, 2022

@russjones @zmb3: Could I get another review for this please? I'd love to get this in.

@russjones
Copy link
Contributor

@wadells Agree with @zmb3 here, both programs are pretty much identical. Let's consolidate them.

As requested by Zac and Russell.
The logic in these two scripts is quite similar, and it will be easier
to maintain a single script instead of two.

This save us ~100 sloc too.
@wadells
Copy link
Contributor Author

wadells commented Jan 13, 2022

@wadells Agree with @zmb3 here, both programs are pretty much identical. Let's consolidate them.

Done. See 4ebffe9 and c0bb0ad.

@wadells wadells requested a review from fheinecke January 14, 2022 02:09
@wadells wadells enabled auto-merge (squash) January 14, 2022 03:35
@wadells wadells changed the title Conditionally publish deb packages to https://deb.releases.teleport.dev/ stable main Conditionally publish deb packages Jan 14, 2022
@wadells wadells merged commit 8540533 into master Jan 14, 2022
@wadells wadells deleted the walt/deb branch January 14, 2022 03:52
wadells added a commit that referenced this pull request Jan 14, 2022
This patch makes a couple changes:

  1. deb archives are not published to apt if they're not the latest
     release ever
  2. both rpm and deb archives are no longer published to yum / apt if
     they contain any pre-release indicator or build metadata
  3. nothing is published if the commit isn't tagged.

Contributes to #8166

(cherry picked from commit 8540533)
wadells added a commit that referenced this pull request Jan 20, 2022
This patch makes a couple changes:

  1. deb archives are not published to apt if they're not the latest
     release ever
  2. both rpm and deb archives are no longer published to yum / apt if
     they contain any pre-release indicator or build metadata
  3. nothing is published if the commit isn't tagged.

Contributes to #8166

(cherry picked from commit 8540533)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants