Skip to content

Latest commit

 

History

History
617 lines (477 loc) · 25.8 KB

release-git-lfs.md

File metadata and controls

617 lines (477 loc) · 25.8 KB

Releasing Git LFS

The core team of Git LFS maintainers publishes releases on a cadence of their determining.

Release Naming

We follow Semantic Versioning standards as follows:

  • MAJOR releases are done on a scale of 2-4 years. These encompass breaking, incompatible API changes, or command-line interface changes that would cause existing programs or use-cases scripted against Git LFS to break.

  • MINOR releases are done on a scale of 2-6 months. These encompass new features, bug fixes, and other "medium"-sized changes into a semi-regular release schedule.

  • PATCH releases are done on the scale of weeks to months. These encompass critical bug fixes, but lack new features. They are amended to a MINOR release "series", or, if serious enough (e.g., security vulnerabilities, etc.) are backported to previous versions.

Release Artifacts

We package several artifacts for each tagged release. They are:

  1. git-lfs-@{os}-v@{release}-@{arch}.tar.gz for the following values:

    operating system architecture
    git-lfs-darwin-amd64-v@{version}.tar.gz darwin amd64
    git-lfs-darwin-arm64-v@{version}.tar.gz darwin arm64
    git-lfs-freebsd-386-v@{version}.tar.gz freebsd 386
    git-lfs-freebsd-amd64-v@{version}.tar.gz freebsd amd64
    git-lfs-linux-386-v@{version}.tar.gz linux (generic) 386
    git-lfs-linux-amd64-v@{version}.tar.gz linux (generic) amd64
    git-lfs-linux-arm-v@{version}.tar.gz linux (generic) arm
    git-lfs-linux-arm64-v@{version}.tar.gz linux (generic) arm64
    git-lfs-linux-loong64-v@{version}.tar.gz linux (generic) loong64
    git-lfs-linux-ppc64le-v@{version}.tar.gz linux (generic) ppc64le
    git-lfs-linux-riscv64-v@{version}.tar.gz linux (generic) riscv64
    git-lfs-linux-s390x-v@{version}.tar.gz linux (generic) s390x

  2. git-lfs-windows-v@{release}-@{arch}.zip for the following values:

    operating system architecture
    git-lfs-windows-386-v@{version}.zip windows 386
    git-lfs-windows-amd64-v@{version}.zip windows amd64
    git-lfs-windows-arm64-v@{version}.zip windows arm64

  3. git-lfs-windows-v@{release}.exe, a signed Windows installer that contains copies of both -x86 and -x64 copies of Git LFS.

  4. *.deb, and *.rpm packages for all of the distributions named in script/packagegcloud.rb.

Development Philosophy

We do all major development on the main branch, and assume it to be passing tests at all times. New features are added via the feature-branch workflow, or (optionally) from a contributor's fork.

This is done so that main can progress and grow new features, while historical releases such as vM.N.0 can receive bug fixes as they are applied to main, eventually culminating in a vM.N.1 (and so on) release.

Building a release

Let release vM.N.P denote the version that we are releasing.

When P is equal to zero, we say that we are releasing a MINOR version of Git LFS in the vM.N-series, unless N is also equal to zero, in which case we are releasing a MAJOR version. Conversely, if P is not equal to zero, we are releasing a PATCH version.

  1. Upgrade the Go version used in the git-lfs/build-dockers repository to the latest available PATCH release with the same major and minor version numbers as the Go version used in this repository's GitHub Actions workflows.

  2. Write the release notes and do the housekeeping required to indicate a new version. For a MAJOR or MINOR version, we start with a main branch which is up to date with the latest changes from the remote and then checkout a new release-next branch from that base.

    If we are releasing a PATCH version, we create a release-M.N branch with cherry-picked merges from the main branch, as described in the instructions below, and then checkout the release-next branch from that base.

    We next perform the following steps to prepare the release-next branch:

    • Run script/changelog and categorize each merge commit as a feature, bug fix, miscellaneous change, or a change to be skipped and ignored. Ensure that your ~/.netrc credentials are up-to-date in order to make requests to the GitHub API, or use a GITHUB_TOKEN environment variable.

      The changelog script will write a portion of the new CHANGELOG to stdout, which you should copy and paste into CHANGELOG.md, along with an H2-level heading containing the new version and the expected release date. This heading should be consistent with the existing style in the document.

      For a MAJOR release, use script/changelog v(M-1).L.0...HEAD, where (M-1) is the previous MAJOR release number and L is the final MINOR release number in that series. For a MINOR release, use script/changelog vM.(N-1).0...HEAD, where (N-1) is the previous MINOR release number, and for a PATCH release, use script/changelog --patch vM.N.(P-1)...HEAD, where (P-1) is the previous PATCH release number.

      • Optionally write 1-2 paragraphs summarizing the release and calling out community contributions.

      • Call out any changes in the operating system versions required by the new release, as well as any differences in the set of Linux platforms for which we build release packages. Check for any new platform requirements from the version of Go in use.

      • If we are releasing a MAJOR or MINOR version and not a PATCH, and if the most recent non-PATCH release was followed by a series of one or more PATCH releases, include any changes listed in the CHANGELOG of that series' release branch in the new release's CHANGELOG. (For a new MAJOR version, the prior release branch would be named release-(M-1).L, following the terminology defined above, while for a new MINOR version the prior release branch would be named release-M.(N-1).)

    • Run script/update-version vM.N.P to update the version number in all of the relevant files. Note that this script requires a version of sed(1) compatible with the GNU implementation.

    • Adjust the date in the debian/changelog entry to reflect the expected release date rather than the current date.

    • Commit all the files changed in the steps above in a single new commit:

      $ git commit -m 'release: vM.N.P'

  3. Push the release-next branch and create a pull request with your changes from the branch. If you're building a MAJOR or MINOR release, set the base to the main branch. Otherwise, set the base to the release-M.N branch.

    • Add the release label to the PR.

    • In the PR description, consider uploading builds for implementers to use in testing. These can be generated from a local tag, which does not need to be signed (but must be annotated). Check that the local version of Go is equivalent to the most recent one used by the GitHub Actions workflows for the release branch, which may be different from that used on the main branch. For a PATCH release in particular you may need to downgrade your local Go version. The build artifacts will be placed in the bin/releases directory and may be uploaded into the PR from there:

      $ git tag -m vM.N.P-pre vM.N.P-pre
$ make release
$ ls bin/releases

    • Notify the @git-lfs/releases and @git-lfs/implementers teams, collections of humans who are interested in Git LFS releases.

    • Ensure that the normal Continuous Integration workflow for PRs that runs automatically in GitHub Actions succeeds fully.

    • As the GitHub Actions release workflow will not run for PRs, consider creating an annotated tag with the -pre suffix and pushing the tag, which will trigger a run of the release workflow that does not upload artifacts to Packagecloud. Alternatively, in a private clone of the repository, create such a tag from the release-next branch plus one commit to change the repository name in script/upload, and push the tag so Actions will run the release workflow. Ensure that the workflow succeeds (excepting the Packagecloud upload step, which will be skipped).

  4. Once approved and verified, merge the pull request you created in the previous step. Locally, create a GPG-signed tag on the merge commit called vM.N.P:

    $ git show -q --pretty=%s%n%b HEAD
Merge pull request #xxxx from git-lfs/release-next
release: vM.N.P

$ git tag -s vM.N.P -m vM.N.P

$ git describe HEAD
vM.N.P

$ git show -q --pretty=%s%d%n%b vM.N.P
tag vM.N.P
Tagger: ...

vM.N.P
-----BEGIN PGP SIGNATURE-----
...
-----END PGP SIGNATURE-----
Merge pull request #xxxx from git-lfs/release-next (tag: vM.N.P)
release: vM.N.P

  5. Push the tag, via:

    $ git push origin vM.N.P

    Validate the pending production environment deployment rule in the UI of the GitHub management application for macOS Developer ID certificates.

    This will kick off the process of building the release artifacts. This process will take somewhere between 45 minutes and an hour. When it's done, you'll end up with a set of release assets available for download, and Linux RPM and Debian packages for the new release will be available on Packagecloud.

  6. Download the release-assets archive file from the "Artifacts" section of the "Summary" page for the most recent run of the release GitHub Actions workflow, and then unpack the file at the top level of the repository:

    $ rm -rf bin/releases
$ unzip /path/to/release-assets.zip

    The release assets should now be present in the bin/releases directory. The script/upload utility will create a new draft release announcement and then upload the release assets and attach them to the announcement:

    $ script/upload --skip-verify vM.N.P

    Note that this script requires GnuPG, with your signing key configured, as well as Ruby 3.x, the OpenSSL Ruby gem, and several other tools, including the GNU coreutils version of b2sum(1). You will need to provide your GitHub credentials in your ~/.netrc file or via a GITHUB_TOKEN environment variable.

  7. Finalize the release process using the same script/upload utility but with the --finalize option. The script will add GPG signatures to the hashes and sha256sums files and then upload the signed files:

    $ script/upload --finalize vM.N.P

    If you want to inspect the data before approving it, pass the --inspect option, which will drop you to a shell and let you look at things. If the shell exits successfully, the build will be signed; otherwise, the process will be aborted.

  8. Publish the release on GitHub, assuming it looks correct.

  9. Move any remaining items out of the milestone for the current release to a future release and close the milestone.

  10. Update the _config.yml file in git-lfs/git-lfs.github.com, similar to the following:

    --- _config.yml
+++ _config.yml
@@ -1,7 +1,7 @@
 # Site settings
 title: "Git Large File Storage"
 description: "Git Large File Storage (LFS) replaces large files such as audio samples, videos, datasets, and graphics with text pointers inside Git, while storing the file contents on a remote server like GitHub.com or GitHub Enterprise."
-git-lfs-release: M.(N-1).0
+git-lfs-release: M.N.0

 url: "https://git-lfs.com"

  11. If Homebrew does not automatically update within a few hours, create a GitHub PR to update the Homebrew formula for Git LFS with the brew bump-formula-pr command on a macOS system. The SHA-256 value should correspond with the packaged artifact containing the new release's source files which is available at the given URL:

    $ brew tap homebrew/core
$ brew bump-formula-pr \
    --url https://github.com/git-lfs/git-lfs/releases/download/vM.N.P/git-lfs-vM.N.P.tar.gz \
    --sha256 <SHA-256> \
    git-lfs

Building PATCH versions

When building a PATCH release, we cherry-pick merges from main to the vM.N release branch, creating the branch first if it does not exist, and then use that branch as the base for the PATCH release.

  1. Upgrade or downgrade the Go version used in the git-lfs/build-dockers repository to the latest available PATCH release with the same major and minor version numbers as the Go version used to build the Git LFS vM.N.(P-1) release.

  2. If the release-M.N branch does not already exist, create it from the corresponding MINOR release tag (or MAJOR release tag, if no MINOR releases have been made since the last MAJOR release):

    $ git checkout -b release-M.N vM.N.0

    If the release branch already exists because this is not the first PATCH release for the given MINOR (or MAJOR) release, simply checkout the release-M.N branch, and ensure that you have the latest changes from the remote.

  3. Gather a set of potential candidates to backport to the release-M.N branch with:

    $ git log --merges --first-parent vM.N.(P-1)...main

  4. For each merge that you want to backport, run:

    $ git cherry-pick -m1 <SHA-1>

    This will cherry-pick the merge onto your release branch, using the -m1 option to specify that the first parent of the merge corresponds to the mainline.

  5. Then follow the guidelines above, using the release-M.N branch as the base for the new PATCH release.

Building security PATCH versions

In our public security policy we request that potential vulnerabilities in the Git LFS client be reported to us via email. Users sometimes also choose to open draft GitHub security advisories, although we do not encourage that option.

If we determine that such a report is valid, we develop a PATCH release version of the client with a remediation for the security issue, following the steps below.

  1. Open a new draft security advisory, fill out the relevant details in the template, and request a CVE identifier.

  2. Create a temporary private fork of this repository from the draft security advisory page, and use the fork to develop a resolution of the vulnerability.

  3. Create two PRs in the private fork, one to remediate the vulnerability in the main development branch of the public git-lfs/git-lfs repository, and one to do so in the appropriate release-M.N branch.

    The PR which targets the release branch should have a final, extra commit which adds an entry for the new PATCH release version in the CHANGELOG.md file, describing the release and the security fix it contains. This commit should also include the changes generated by the script/update-version vM.N.P command, as per the second step in our standard release process.

    If the release-M.N branch does not exist in the public repository, create it as per our regular PATCH release process, and then create the second PR in the private fork against the new branch. Be prepared to proceed relatively quickly, since the appearance of the new branch serves as a public notice that we may be publishing a PATCH release shortly.

  4. From the draft security advisory page, use the "Merge pull requests" option to merge both PRs simultaneously.

    Note that both the release branch and the main branch in the private fork must be up-to-date with the corresponding branches in the public repository in order for the GitHub UI to determine whether the PRs have no merge conflicts.

  5. Publish the security advisory.

  6. Follow our standard release process, starting with the step in which we create a GPG-signed tag named vM.N.P on the merge commit to the release-M.N branch.

  7. After publication of the new release, update the git-lfs.com home page with a banner message regarding the security PATCH release. Place the banner at the top of the _includes/home/secondary.html file in the git-lfs/git-lfs.github.com repository. Use the GHSA identifier from the security advisory, and remove any previous security message banners.

    <div class="column">
  <a class="dot-com-announcement" data-ga-params="banner"
    href="https://github.com/git-lfs/git-lfs/security/advisories/GHSA-abcd-1234-wxyz">
    Git LFS security update: <span>All users should update to M.N.P or newer</span>.</a>
  </a>
</div>

  8. Create a new discussion in the "Announcements" section of the GitHub discussion forum describing the security PATCH release, and pin the discussion for all categories in the forum.

Building security PATCH versions under embargo

When coordinating the release of a security patch with one or more other projects, we must not follow the processes described above as they depend on the GitHub Actions release workflow in our public git-lfs/git-lfs repository. Instead, we use a private repository with a modified release workflow to build the binaries and packages for our new version so that we can share them with the other projects in advance of the coordinated release date.

  1. Open a draft security advisory, apply for a CVE identifier, and create a temporary private fork from the advisory page. Develop a resolution of the vulnerability in the private fork. These initial steps are the same as those of our non-embargoed security PATCH release process.

  2. Create one PR in the private fork to remediate the vulnerability in the main development branch of the public repository.

  3. Create a branch in the private fork from the appropriate release-M.N branch, or if that does not yet exist, from the appropriate vM.N.0 tag, and add the necessary changes to remediate the vulnerability.

    This branch should have one extra, final commit which adds an entry for the new PATCH release version in the CHANGELOG.md file, describing the release and the security fix it contains. This commit should also include the changes generated by the script/update-version vM.N.P command, as per the second step in our standard release process.

  4. Pull this branch from the private fork of the public repository and push it into a separate private repository which has a fresh copy of the public git-lfs/git-lfs repository, and for which our full CI and release GitHub Actions workflows are configured and enabled.

    Note that the private fork created from the draft security advisory will not execute GitHub Actions jobs, and so we require the use of a separate private repository to run a modified release workflow.

  5. If the release-M.N branch does not exist, create it in the separate private repository from the vM.N.0 tag, as described in the second step of our regular PATCH release process.

  6. Merge the branch containing the security fix and the commit with the new CHANGELOG.md entry (and script/update-version vM.N.P changes) into the release-M.N branch, and sign the merge commit with your GPG key:

    $ git checkout release-M.N
$ git merge --no-ff -S \
    -m "Merge pull request from GHSA-abcd-1234-wxyz" \
    -m "release: M.N.P" \
    branch-with-fix-and-changelog

    Use the GHSA identifier from the draft security advisory in the merge commit's description.

  7. Create a GPG-signed tag named vM.N.P on the merge commit, using the same command from the equivalent step of our standard release process:

    $ git tag -s vM.N.P -m vM.N.P

  8. Check out the main branch of the private repository and make the following revisions to the .github/workflows/release.yml file, and then push these changes back to the private repository's main branch.

    First, change the on value to workflow_dispatch:

    -on:
-  push:
-    tags: '*'
+on: workflow_dispatch

    Set the version of Go in each matrix context to the same version used in the Dockerfiles from our git-lfs/build-dockers repository.

    Replace the ${{ github.ref }} context wherever it appears with the vM.N.P tag:

    -    ref: ${{ github.ref }}
+    ref: vM.N.P

    Revise the build-docker and build-docker-arm jobs so they do not upload the Linux packages generated by the docker/run_dockers.bsh script to Packagecloud by running the script/packagecloud.rb utility. Change the jobs to instead upload the Linux packages as job artifacts; for instance, for the build-docker job:

        - uses: actions/upload-artifact@v4
      with:
        name: docker-assets
        path: |
          repos/**/*.deb
          repos/**/*.rpm

    Make sure to use distinct artifact names for each of these jobs, i.e., docker-assets and docker-arm-assets.

  9. Push the vM.N.P tag to the private repository, and then cancel the GitHub Actions job which runs from the release workflow.

    (This workflow will run because the tag includes our normal GitHub Actions workflow definitions without the changes made in the previous step, since those exist only on the main branch, and we do not want to include these temporary workflow changes in the security release itself.)

    Confirm that the CI workflow job succeeds.

  10. From the private repository's GitHub Actions page, manually dispatch the release workflow.

  11. When the manually-dispatched job is complete, download the release-assets, docker-assets, and docker-arm-assets archive files from the "Artifacts" section of the job's "Summary" page.

    Share the relevant binaries from the archive files with the other collaborating projects during the embargo period.

  12. Before the embargo is due to the lifted, unpack the release-assets archive file at the top level of this repository and use the script/upload utility to create a draft release announcement and attach the release assets to it, just as for a standard release:

    $ rm -rf bin/releases
$ unzip /path/to/release-assets.zip

$ script/upload --skip-verify vM.N.P

  13. When the embargo is lifted, use the "Merge pull requests" option on the draft security advisory page to merge the PR with the security fix into the main branch of the public repository.

    Note that the main branch in the private fork must be up-to-date with the corresponding branch in the public repository in order for the GitHub UI to determine whether the PR has no merge conflicts.

  14. Publish the security advisory.

  15. Pull the release-M.N branch and vM.N.P tag from the private repository where the release workflow was run manually, and push them into the public git-lfs/git-lfs repository.

    Immediately cancel the GitHub Actions job which runs from the release workflow in the public repository.

    Note that cancelling the release workflow job is important, since it will otherwise build new versions of the RPM and Debian Linux packages and publish them to Packagecloud.

  16. Upload to Packagecloud the RPM and Debian Linux packages in the docker-assets and docker-arm-assets archive files created earlier by the manual release workflow job.

    Note that the script/packagecloud.rb utility requires the PACKAGECLOUD_TOKEN environment variable to contain the current Packagecloud account credential token.

    $ mkdir repos
$ unzip -d repos /path/to/docker-assets.zip
$ unzip -d repos /path/to/docker-arm-assets.zip

$ gem install packagecloud-ruby
$ PACKAGECLOUD_TOKEN="<token>" script/packagecloud.rb

  17. Finalize the release process using the script/upload utility with the --finalize option. The script will add GPG signatures to the hashes and sha256sums files and then upload the signed files:

    $ script/upload --finalize vM.N.P

  18. Publish the release announcement.

  19. Update the _config.yml file in the git-lfs/git-lfs.github.com repository with the new M.N.P release version and push the change, as described in the final steps of our standard release process.

    In addition, update the _includes/home/secondary.html file with a banner message regarding the security PATCH release, and push the change.

  20. Create a new discussion in the "Announcements" section of the GitHub discussion forum describing the security PATCH release, and pin the discussion for all categories in the forum.