Automated Release scripts #2174

ognyanstoimenov · 2025-02-11T09:29:07Z

Reference Issues/PRs

What does this implement or fix?

Currently versions are released by manually running the tag.yml workflow, which releases selected branch with a specified version.
This extends that with scripts that call the tag.yml with version to release and branch to release it from.

Flow:

Main entry point - automated_release.full_version_cron - on a schedule (or manual trigger) releases the latest release branch. Release branches follow the naming convention X.Y.Z so the release produced will be vX.Y.Z
When a release is created - automated_release.create_release_branch is triggered which calculates what the next release version should be based on commits in master and create that release branch from which would be the base of the next release. Immediately releases it as rc0 (calls tag.yml to release <new_branch_name>rc0)
On every commit in release branch - automated_release.rc_version releases next rc version of that branch (calls tag.yml to release <release_branch_name>rc1 or rc2, etc)

Marking PR's as `patch`, `minor`, `major`

New PR template:

## Change Type (Required)
- [ ] **Patch** (Bug fix or non-breaking improvement)
- [ ] **Minor** (New feature, but backward compatible)
- [ ] **Major** (Breaking changes)
- [ ] **Cherry pick**

On every PR opened/edited to master automated_release.parse_pr_template - parses new template and adds apropriate label which in future gets parsed by automated_release.create_release_branch specified above

Proof of working/demo (~7min, watch in 1.5x speed)

1080.Screen.Recording.2025-02-13.155706.mp4

Any other comments?

Checklist

Checklist for code changes...

Have you updated the relevant docstrings, documentation and copyright notice?
Is this contribution tested against all ArcticDB's features?
Do all exceptions introduced raise appropriate error messages?
Are API changes highlighted in the PR description?
Is the PR labelled as enhancement or bug so it appears in autogenerated release notes?

G-D-Petrov

Looks pretty good, I just have a couple of questions

G-D-Petrov · 2025-02-13T15:20:46Z

.github/workflows/automated_release.create_release_branch.yml

+    permissions:
+      contents: write
+      pull-requests: write
+      repository-projects: write


Do we really need all of these permissions?

Thought they are needed for the downstream jobs, but just tested and seems like they are not necessary, will remove

G-D-Petrov · 2025-02-13T15:25:00Z

.github/workflows/automated_release.full_version_cron.yml

+        uses: oprypin/find-latest-tag@v1
+        with:
+          repository: ${{ github.repository }}
+          regex: ^v\d+\.\d+\.\d+rc\d+$


This is called with different params than the ones in create-next-release-branch.
Maybe equalize them

They are slightly different in meaning. This one searches for the latest rc tag, and later strips 'rc'. while the other one searches for the latest full release and later increments the version

maxim-morozov · 2025-02-14T11:02:28Z

.github/workflows/automated_release.create_release_branch.yml

+          echo Labels: $labels
+          increment_type="patch"
+          case "$labels" in
+            *"${{ env.MAJOR_LABEL }}"*) increment_type="major"; echo "Major changes found! Should increase major version." ;;


How this would work when we have mix of major, minor and patch between releases? Say something like when we cut the release candidate branch, master had only minor changes. But then during the week, master had major version changes. I wonder if we need to think about it?

maxim-morozov · 2025-02-14T11:03:44Z

.github/workflows/automated_release.full_version_cron.yml

+name: "Automated Release: Release next full version from release branch"
+
+on:
+  # schedule:


should we enable this? Or you want to test it manually first?

Let's test manually first and if all is ok, I will enable it

poodlewars · 2025-02-19T17:59:47Z

.github/PULL_REQUEST_TEMPLATE.md

@@ -3,6 +3,12 @@

 #### What does this implement or fix?

+## Change Type (Required)


Can we just enforce these with labels on the PR rather than these checkboxes?

It is possible yes, maybe even clearer. I will enforce it so we can't merge without the labels. Also I think we can use the same set of labels for releases and release notes, so that we don't have e.g. both minor and feature but just the one that would be used for both release versions and release notes

poodlewars · 2025-02-19T18:05:49Z

.github/workflows/automated_release.rc_version.yml

+on:
+  push:
+    branches:
+      - '[0-9]+.[0-9]+.[0-9]+'


I think we need a stricter naming convention for these branches - otherwise someone will create a branch 65.11.22 and it will get released. It might be possible to configure the repo to block creation of these branches other than by these workflows

Yes, I agree. I thought we could use x.y.z-<branch cut sha> as naming convention but it looked ugly. So blocking such creations should be fine. I will try to configure that as well as some merging restrictions in release branches

poodlewars · 2025-02-19T18:08:23Z

.github/workflows/automated_release.full_version_cron.yml

How will we patch older versions with this? Say we have,

v5.5.5rc3 -> we want to release this to 5.5.3
v4.4.2rc1 -> we want to release this to 4.4.2

I don't understand what you mean. Why would we patch an older version? If we have v5.5.5rc3 or rc4 etc, when it is time we release v5.5.5 and that's it for that version. If we find something is wrong after the full release and we want to patch it would be in the next patch version 5.5.6

#### Reference Issues/PRs  * New workflow that on trigger creates a dev release from current master * Calculates what next version would be and names the release `x.y.z.dev0+<master_sha>` * Release scripts introduced in #2174 had some permissions/token issues. This fixes them. * Not working (5.2.4 branch, check annotations); https://github.com/man-group/ArcticDB/actions/runs/13397301934 https://github.com/man-group/ArcticDB/actions/runs/13409868194 * Working (5.2.4 after applying this): * https://github.com/man-group/ArcticDB/actions/runs/13410089783 #### What does this implement or fix? ## Change Type (Required) - [x] **Patch** (Bug fix or non-breaking improvement) - [ ] **Minor** (New feature, but backward compatible) - [ ] **Major** (Breaking changes) - [ ] **Cherry pick** #### Any other comments? #### Checklist <details> <summary> Checklist for code changes... </summary> - [ ] Have you updated the relevant docstrings, documentation and copyright notice? - [ ] Is this contribution tested against [all ArcticDB's features](../docs/mkdocs/docs/technical/contributing.md)? - [ ] Do all exceptions introduced raise appropriate [error messages](https://docs.arcticdb.io/error_messages/)? - [ ] Are API changes highlighted in the PR description? - [ ] Is the PR labelled as enhancement or bug so it appears in autogenerated release notes? </details>

#### Reference Issues/PRs  #### What does this implement or fix? Currently versions are released by manually running the tag.yml workflow, which releases selected branch with a specified version. This extends that with scripts that call the tag.yml with version to release and branch to release it from. ### Flow: * **Main entry point** - `automated_release.full_version_cron` - on a schedule (or manual trigger) releases the latest release branch. Release branches follow the naming convention `X.Y.Z` so the release produced will be `vX.Y.Z` * **When a release is created** - `automated_release.create_release_branch` is triggered which calculates what the next release version should be based on commits in `master` and create that release branch from which would be the base of the next release. Immediately releases it as `rc0` (calls `tag.yml` to release `<new_branch_name>rc0`) * **On every commit in release branch** - `automated_release.rc_version` releases next rc version of that branch (calls `tag.yml` to release `<release_branch_name>rc1` or rc2, etc) ### Marking PR's as `patch`, `minor`, `major` * **New PR template**: ``` ## Change Type (Required) - [ ] **Patch** (Bug fix or non-breaking improvement) - [ ] **Minor** (New feature, but backward compatible) - [ ] **Major** (Breaking changes) - [ ] **Cherry pick** ``` * **On every PR opened/edited to master** `automated_release.parse_pr_template` - parses new template and adds apropriate label which in future gets parsed by `automated_release.create_release_branch` specified above ## Proof of working/demo (~7min, watch in 1.5x speed) https://github.com/user-attachments/assets/94705eb1-dd76-4397-8e2e-44828fc01ceb #### Any other comments? #### Checklist <details> <summary> Checklist for code changes... </summary> - [ ] Have you updated the relevant docstrings, documentation and copyright notice? - [ ] Is this contribution tested against [all ArcticDB's features](../docs/mkdocs/docs/technical/contributing.md)? - [ ] Do all exceptions introduced raise appropriate [error messages](https://docs.arcticdb.io/error_messages/)? - [ ] Are API changes highlighted in the PR description? - [ ] Is the PR labelled as enhancement or bug so it appears in autogenerated release notes? </details>

#### Reference Issues/PRs  * New workflow that on trigger creates a dev release from current master * Calculates what next version would be and names the release `x.y.z.dev0+<master_sha>` * Release scripts introduced in #2174 had some permissions/token issues. This fixes them. * Not working (5.2.4 branch, check annotations); https://github.com/man-group/ArcticDB/actions/runs/13397301934 https://github.com/man-group/ArcticDB/actions/runs/13409868194 * Working (5.2.4 after applying this): * https://github.com/man-group/ArcticDB/actions/runs/13410089783 #### What does this implement or fix? ## Change Type (Required) - [x] **Patch** (Bug fix or non-breaking improvement) - [ ] **Minor** (New feature, but backward compatible) - [ ] **Major** (Breaking changes) - [ ] **Cherry pick** #### Any other comments? #### Checklist <details> <summary> Checklist for code changes... </summary> - [ ] Have you updated the relevant docstrings, documentation and copyright notice? - [ ] Is this contribution tested against [all ArcticDB's features](../docs/mkdocs/docs/technical/contributing.md)? - [ ] Do all exceptions introduced raise appropriate [error messages](https://docs.arcticdb.io/error_messages/)? - [ ] Are API changes highlighted in the PR description? - [ ] Is the PR labelled as enhancement or bug so it appears in autogenerated release notes? </details>

ognyanstoimenov force-pushed the ostoimenov/automated_release branch from 4df65ee to 1e842af Compare February 12, 2025 23:31

ognyanstoimenov changed the title ~~Release scripts~~ Automated Release scripts Feb 12, 2025

Release scripts

14f9ce4

ognyanstoimenov force-pushed the ostoimenov/automated_release branch from 1e842af to 14f9ce4 Compare February 13, 2025 14:00

ognyanstoimenov marked this pull request as ready for review February 13, 2025 14:45

G-D-Petrov reviewed Feb 13, 2025

View reviewed changes

Remove unneeded permissions

22814a9

G-D-Petrov approved these changes Feb 13, 2025

View reviewed changes

maxim-morozov reviewed Feb 14, 2025

View reviewed changes

maxim-morozov approved these changes Feb 14, 2025

View reviewed changes

ognyanstoimenov merged commit 12b9acd into master Feb 14, 2025
152 checks passed

ognyanstoimenov deleted the ostoimenov/automated_release branch February 14, 2025 17:05

This was referenced Feb 17, 2025

Include wheels in releases #2180

Merged

Developmental releases on demand #2190

Merged

poodlewars reviewed Feb 19, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Automated Release scripts #2174

Automated Release scripts #2174

ognyanstoimenov commented Feb 11, 2025 •

edited

Loading

G-D-Petrov left a comment

G-D-Petrov Feb 13, 2025

ognyanstoimenov Feb 13, 2025

G-D-Petrov Feb 13, 2025

ognyanstoimenov Feb 13, 2025

maxim-morozov Feb 14, 2025

maxim-morozov Feb 14, 2025

ognyanstoimenov Feb 14, 2025

poodlewars Feb 19, 2025

ognyanstoimenov Feb 20, 2025

poodlewars Feb 19, 2025 •

edited

Loading

ognyanstoimenov Feb 20, 2025 •

edited

Loading

poodlewars Feb 19, 2025

ognyanstoimenov Feb 20, 2025

		@@ -3,6 +3,12 @@

		#### What does this implement or fix?

		## Change Type (Required)

Automated Release scripts #2174

Automated Release scripts #2174

Conversation

ognyanstoimenov commented Feb 11, 2025 • edited Loading

Reference Issues/PRs

What does this implement or fix?

Flow:

Marking PR's as patch, minor, major

Proof of working/demo (~7min, watch in 1.5x speed)

Any other comments?

Checklist

G-D-Petrov left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

poodlewars Feb 19, 2025 • edited Loading

Choose a reason for hiding this comment

ognyanstoimenov Feb 20, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ognyanstoimenov commented Feb 11, 2025 •

edited

Loading

Marking PR's as `patch`, `minor`, `major`

poodlewars Feb 19, 2025 •

edited

Loading

ognyanstoimenov Feb 20, 2025 •

edited

Loading