Skip to content

Making a release

Jump to bottom
brandon s allbery kf8nh edited this page Oct 10, 2024 · 254 revisions

Release checklist

Pick one of section A or B below depending on what kind of release you're making. Either section will call out to section C for common steps. You don't need to go over section C separately.

A Making a minor or patch release (AKA major-not-forcing release)

These releases should be mainly composed of bugfix backports from master. The API exposed by the libraries must not change in a breaking way for current consumers. The cabal file format (syntax & grammar) must not change at all.

A.1 Before the release

  1. Pre-flight Checks: See Preflight checks
  2. Changelog generation: See CHANGELOGS
  3. Bump version numbers: See Bumping version numbers
  4. Upload release candidates: See Uploading package release candidates to Hackage
  5. Publish artifacts: See Publishing the artifacts
  6. Publish everything else: See Publishing everything else
  7. Tweak things as needed, merge patches to fix regressions
  8. Re-upload the candidates on Hackage based on the latest tag, and publish them

A.2 After the release

  1. prepare a list of contributors for the release, e.g., git log --use-mailmap --pretty=format:%an Cabal-v3.6.1.0...Cabal-v3.8.1.0 | sort -u and don't forget to mention "reviewers, QA testers, devops and others"
  2. advertise on discourse, haskell-cafe and cabal-devel and in particular, mention:
    • How to install using ghcup
    • Changelogs
    • Fixed regression compared to the previous version
    • Regressions still unfixed
    • How great our community is and how we love and cherish collaborators
  3. ask kind souls to cross-post to Mastodon, Matrix (#haskell-tooling and #haskell-language-server), Discord, Twitter, Reddit. Copy the links to #hackage so that others can upvote
  4. Move the tickets that haven't made it to the release to the next one.
  5. party 🥳

B Making a major release (AKA major-forcing release)

B.1 Before the release
  • -1. Consider upgrading fourmolu and reformating the codebase with the new version of it. We want to do it at least every two major releases of GHC/Cabal. Important to do it before the cut (the next step) so that backports don't bump into formatting conflicts.
  1. Cut a new branch for the new major release named EPOCH.MAJOR (like 3.10 or 3.12).
  2. Pre-flight Checks: See Preflight checks
  3. Add the new SPDX License List data (warning: this is a breaking PVP change, so should be done on major versions only!) This is not optional! (It is constantly changing as new licenses are added, and if we don't keep up we will get bug reports for unsupported licenses.)
  4. Changelog generation: See CHANGELOGS
  5. Bump version numbers: See Bumping version numbers
  6. Bump the versions again, this time on master, to the next odd version (3.11 -> 3.13).
  7. Bump copyright years everywhere on branch master so that it doesn't have to be repeated on each branch and backport the PR to your release branch
  8. Check the corresponding Release checklist for GHC X.X ticket (e.g., https://github.com/haskell/cabal/issues/9729), if there is one, perform the actions, tick the boxes and make sure nothing is missing that should be done before the packages are published on Hackage.
  9. Upload release candidates: See Uploading package release candidates to Hackage
  10. Publish artifacts: See Publishing the artifacts
  11. Publish everything else: See Publishing everything else
  12. Tweak things as needed, look for new issues labelled "regression in ...", merge patches that fix regressions
  13. Re-upload the candidates on Hackage based on the latest tag, and publish them

WARNING: avoid naming your working branches for PRs with names starting with the branch name, as they will be matched by our branch protection rules and you will run into restrictions as a result.

B.2 After the release
  1. Re-run make bootstrap-jsons and submit a PR to master, if the plans are not up to date (e.g., because package bounds have not been bumped soon enough or there were technical difficulties).
  2. Move the tickets that haven't made it to the release to the next one.
  3. prepare a list of contributors for the release, e.g., git log --use-mailmap --pretty=format:%an Cabal-v3.6.1.0...Cabal-v3.8.1.0 | sort -u and don't forget to mention "reviewers, QA testers, devops and others"
  4. advertise on blog.haskell.org (how to install using ghcup (ghcup install cabal --url https://downloads.haskell.org/~cabal/cabal-install-3.8.1.0-rc1/cabal-install-3.8.1.0-x86_64-linux-deb10.tar.xz 3.8.1.0), changelogs, how great our community is and how we love and cherish collaborators, known important bugs and that if anybody is keen to see a particular bug go before 3.8.1.0, the window is now very narrow but we will try to help the interested person squash the bug)
  5. ask kind souls to cross-post to haskell-cafe cabal-devel, Discourse, Discord, cabal-devel mailing list, Twitter, Reddit. Copy the links to #hackage so that others can upvote
  6. party 🥳

C Common steps

C.1 Preflight checks

The goal is to create or backport all needed PRs in this step so that the list of PRs is finalized before changelog generation and so no PRs are missed in the changelog. CI (github and gitlab) PRs are exempt since they emerge from much later steps to do with binary builds, but it's fine because they are not recorded in the changelog.

  1. Make sure cabal-version of Cabal and Cabal-syntax is not the latest cabal file format version and that it is within the backwards compat range, which extends to 5 years back. Look at the file format changelog to see which cabal-version was released five years ago (the SPDX License List version gives away the date).

  2. Run cabal check in ./Cabal, ./cabal-install, ./Cabal-syntax, ./Cabal-hooks and ./cabal-install-solver` and fix as needed.

  3. Run cabal outdated in ./Cabal, ./cabal-install, ./Cabal-syntax, /Cabal-hooks and ./cabal-install-solver and update as needed.

  4. Commit the changes to master, then backport them to the release branch. You may also want to backport to still-supported branches. (e.g. if we are releasing 3.12, we will commit to master, backport to 3.12 and backport to 3.10.)

C.2 CHANGELOGs

One challenge with changelogs is to manage the duplication of entries between the Cabal and cabal-install packages. Changes that substantially affect both the library API and the behavior of cabal-install shall be reported in both packages, e.g., because some users read only one of the changelogs. On the other hand, if the Cabal change is minor and doesn't alter the API or if the cabal-install behavior is barely changed, then no duplication is necessary. Use your best judgment.

You should wait before all backports are merged (including SPDX licenses) before starting with changelogs. This way you ensure you can sweep all changelogs file in one go.

  1. Prerequisite: Install https://codeberg.org/fgaz/changelog-d

  2. Switch to the release branch.

  3. Create a PR log for the release you are targeting git log --pretty=oneline --no-color --decorate-refs='*' <package>-v<source>..<destination> > <destination>.prlog. For instance: git log --pretty=oneline --no-color --decorate-refs='*' cabal-install-v3.10.2.1..HEAD > 3.10.3.0.prlog

  4. Run:

    # For now, put Cabal-syntax and cabal-install-solver PR changelogs items into Cabal and cabal-install changelogs:
$ changelog-d --long --package Cabal --prlog 3.14.0.0.prlog >> Cabal.changes
$ changelog-d --long --package Cabal-syntax --prlog 3.14.0.0.prlog >> Cabal.changes
$ changelog-d --long --package cabal-install --prlog 3.14.0.0.prlog >> cabal-install.changes
$ changelog-d --long --package cabal-install-solver --prlog 3.14.0.0.prlog >> cabal-install.changes
$ changelog-d --long --package Cabal-hooks --prlog 3.14.0.0.prlog >> Cabal-hooks.changes

    Note: sanity checking. The git log does not affect which PRs are included in the release notes and is only used to generate warnings, which is why changelogs need to be generated on the release branch, where the tiny files in changelog.d/ exactly match what's being released. The warnings about "missing PRs" are not very useful at present (Sep 2024) because changelog-d doesn't understand backports.

  5. Manually integrate the content of these .changes files into the changelog files of the Cabal and cabal-install packages.

  6. Manually deduplicate entries inside individual files if needed. (Different files can contain duplicate entries if a change touches both the library and the tool.)

  7. Add sections for the release to Cabal/ChangeLog.md, Cabal-syntax/ChangeLog.md, cabal-install/changelog, and cabal-install-solver/ChangeLog.md. These should point to the respective changelog files on master.

  8. Add an “Unresolved” section. In this section, put issues that are marked as “regression” in the issue tracker and we did not manage to fix in time for the release, so that users are warned.

    • Between minor releases: all regressions that bubbled up between (minor) releases should be mentioned.
    • Between major releases: use your judgement and mention only regressions which are important. You don’t need to paste the whole list of regressions by default.

  9. Make a PR that adds the new changelogs and cleans up the changelog-d directory (except changelog-d/config), preferably with two separate commits (adding the notes and removing the files, respectively).

  10. Forward-port the changelogs PR to the master branch because it is the authoritative branch for changelogs. All links in changelogs point to it. Note that this should be merged before the actual release so it's already in place.

Note on contents of changelog-d/ directory. Regardless if that's a pre-release or release, all the tiny changelog files from changelog.d/ should be removed. If it's a pre-release, its changelog will be included in the next major version's changelog, so the tiny changelogs are not needed any more. (Beware: there is a file config which must not be removed.)

C.3 Bumping version numbers

In the points below, start by creating PRs that target the branch you're releasing (not master). A single PR can implement many points. Afterward, most of the same tasks need to be performed for the master branch, but usually the PRs can't be forward-ported from 3.12 to master, because version on master branch are different. In case of difficulties, some master tasks can be postponed, especially regenerating boostrap plans, because it's not critical for development on master branch, unlike proper version numbers.

  1. Change the version fields in:
    • Cabal-syntax/Cabal-syntax.cabal
    • Cabal/Cabal.cabal
    • cabal-install-solver/cabal-install-solver.cabal
    • cabal-install/cabal-install.cabal
    • doc/conf.py
    • Cabal/Makefile
  2. Change the dependency constraints on Cabal-syntax and Cabal to the new version where they are needed in the repo (like cabal-testsuite/cabal-testsuite.cabal and bootstrap/cabal-bootstrap-gen.cabal).
    • Don't update the custom-setup section of cabal-testsuite/cabal-testsuite.cabal beyond the version of cabal-install used by CI.
  3. Update the lists of GHC versions in .github/workflows/validate.yml (remember: five-year support window for GHC) and .github/workflows/bootstrap.yml (a few of the most recent GHCs suffice, ask on IRC/Matrix if unsure).
  4. (only for major releases) Update the list of SPDX_LICENSE_VERSIONS in Makefile, if you haven't done already.
  5. Update the list of BOOTSTRAP_GHC_VERSIONS in Makefile (to match what is in .github/workflows/bootstrap.yaml).
  6. Deal with hackage-security. This is a five step process:
    1. Clone hackage-security repository.
    2. Bump Cabal-syntax (and Cabal, if needed) dependency constraints in hackage-security's cabal files.
    3. Test it locally by building hackage-security with the new Cabal-syntax (hackage-security repo has a cabal.project file, you can add Cabal-syntax/ and Cabal/ via packages: directive).
    4. Open a PR with the relevant changes; this can be merged in a speedy fashion (ask devs, and again note that it must be merged manually.) and released (example PR).
    5. Wait for the release on Hackage and then update index-state: in cabal.release.project.
  7. Open a PR bumping the index state in https://github.com/haskell/cabal/blob/master/cabal.release.project (and the relevant release branch) and any other files that fix the index state and ask for volunteers to handle the fallout, if any. (needed to permit the new hackage-security and also any upstream fixes emerging due to testing with the new GHC)
  8. Re-generate the bootstrap build plans (json files) by running make bootstrap-jsons. If the program errors because it did not find your global configuration file, export an environment variable (e.g. export CABAL_CONFIG=~/.config/cabal/config) to fix the error.

C.4 Tagging the release

  1. Make tags:

    Switch to the release branch and

    # --sign uses the default e-mail address’ key in your keyring.
# Use --local-user=<key-id> if you want to pick another key. 
git tag --sign Cabal-vxxxx
git tag --sign Cabal-syntax-vxxxx
git tag --sign cabal-install-vxxxx
git tag --sign cabal-install-solver-vxxxx

  2. Push tags to upstream, e.g.

    # Change `origin` with the appropriate name 
git push origin Cabal-syntax-vxxxx Cabal-vxxxx …

NOTE: The branch is FROZEN once the tags have been pushed. If it becomes necessary for some reason to add additional commits, the tags must be redone and force-pushed and all subsequent steps must be repeated if necessary!

C.5 Uploading package release candidates to Hackage

The following packages need to be uploaded to Hackage:

  • For a Cabal library release:

    • Cabal-syntax
    • Cabal
    • Cabal-hooks (note that this package is versioned separately, so uploading it is only required if there is a new version since the last Cabal library release)

  • For a cabal-install release:

    • The libraries listed above (if not done already)
    • cabal-install-solver
    • cabal-install

We start by first uploading release candidates of all the packages, so that we can check everything is OK before proceeding.

  1. Upload candidates for all the packages using cabal sdist, cabal haddock --haddock-for-hackage … and then cabal upload --documentation … for each package.

  2. Check that the resulting package candidates on Hackage look good:

    • check the main package pages, including version number, exported modules etc,
    • verify that the link to the release notes inside the changelog on Hackage works,
    • check that cross-package references work correctly.

  3. Post the candidates to the #hackage IRC channel to get further feedback.

C.6 Publishing the artifacts

The goal is to build binary artifacts from the tags created earlier, and sign and publish both the artifacts and the tagged source code. Check SHA256SUMS and PGP handbook for cabal developers if you need help with PGP steps.

  1. Wait 30min for Gitlab to synchronise, and start the CI at https://gitlab.haskell.org/haskell/cabal/-/pipelines/new. Pick the appropriate branch, and then wait for the process to complete. For each job, go to their artifacts section, click Browse, then out/. NOTE: the next page shows a link with a filename. This is not the actual artifact! You must click on the file name, then save or save-as the link on the Download button.
  2. manually copy the artifacts to a newly created local directory release/
  3. then copy the cabal sdist tarballs as the source tarballs (not the whole source trees, e.g., no bootstrap files)
  4. create and sign checksums using something like sha256sum * > SHA256SUMS; gpg --detach-sign SHA256SUMS. Midstream distributors rely on them.
  5. Check with gpg --verify SHA256SUMS.sig
  6. Be sure that your public key has been uploaded to a few well-known public keyservers (we recommend https://keyserver.ubuntu.com/)
  7. copy all files from release/ to the downloads site using sftp [email protected], creating the remote release directory first
  8. prod the CDN via curl -X PURGE http://downloads.haskell.org/cabal/ in order to get the tarballs to show at https://downloads.haskell.org/cabal or peek at the new CDN content at https://downloads.haskell.org/cabal/?foo<release_number>, e.g., https://downloads.haskell.org/cabal/?foo3.12.1.0
  9. If the tarballs have been mistakenly uploaded and you need to replace them, be sure to run PURGE on the exact paths.
  10. update the -latest symbolic links via the same sftp tool on https://downloads.haskell.org/cabal
  11. wait (sometimes hours) until CDN kicks in and verify at https://www.haskell.org/cabal/download.html that it looks fine and new links work

C.7 Publishing everything else

  1. someone has been making release artifacts for platforms we don't build for, e.g. freebsd. I don't know what happens if they disappear.

  2. create a ghcup PR to include the new cabal release, such as https://github.com/haskell/ghcup-metadata/pull/28

    • summer 2024 update: ghcup now only accepts updates to the vanilla channel, while the main channel is managed solely by ghcup maintainers, it seems.

  3. add a new version to readthedocs (requires edit access)

    1. first activate the relevant tag
    2. and then run the build
    3. create a tag like 3.12.0+0 (our tags are not legal for readthedocs); check that stable points to the same revision (you can get that from the bottom of the page) as the release docs.

  4. make a minimalist github release (not for pre-releases) Note: make sure you do this before GHA updates its images!

  5. create, review and merge an update PR at https://github.com/haskell/cabal-website, possibly touching only download.html

  6. cd cabal-website; sftp [email protected]

  7. cd cabal

  8. put download.html

  9. exit

SHA256SUMS and PGP handbook for cabal developers

We publish SHA256-checksums for every file that is a part of a release. And we sign the file with the checksums. So, to do sanity-checking around this, you need to, first, check the sums and, second, verify the signature. The first step is this:

sha256sum --check SHA256SUMS

Note that you need to download the SHA256SUMS file along with every binary that you want to check.

Second, check the signature:

  1. Obtain the key of the signatory:

    gpg --keyserver keyserver.ubuntu.com --search-key <dev-name-or-email-or-keyid>

    To verify it worked, you can list the ID of the key: gpg --list-key <email-or-name> (30 hex digits) or list the fingerprint: gpg --fingerprint <email-or-name-or-keyd>.

  2. Verify the the signature of SHA256SUMS. You only need the two files: SHA256SUMS and SHA256SUMS.sig.

    gpg --verify SHA256SUMS.sig

    Note: if gpg complains about a missing key, you failed the previous step.

Important: the key ID should be a part of the release announcement.

When there is a new signatory, we need to sign the key, to expand our network of trust of Cabal devs. Simple steps for mutual key verifications:

  1. make sure it is them (this usually involves a phone-call and a question only they can answer)
  2. download each other public key
  3. read aloud the fingerprints and double check them
  4. gpg --sign-key <their-key-id>
  5. gpg --armor --export <their-key-id> > signed-key.asc.gpg
  6. send the file to them via email
  7. wait for their email
  8. gpg --import <file> this imports the new signature to your key
  9. gpg --send-keys --keyserver keyserver.ubuntu.com <your-key-id>
  10. check the signature is there on https://keyserver.ubuntu.com
  • remove release notes from pre-release a couple of numbers back (not to invalidate too early people's links done not via a tag or a commit)

Todo during a low-pressure release

For pre-releases

Cabal has two kinds of pre-releases: official and unofficial ones. When you announce a prerelease, make sure to point out which kind of prerelease you're announcing.

  • Always create a Git tag for the prerelease.

  • Don't publish Hackage candidates.

  • Binary artifacts:

    • Official pre-releases require an entry on the haskell.org/cabal/downloads site directory for posterity.

    • Unofficial pre-release artifacts are hosted by Gitlab alone and will disappear at some point.

  • Don't mention the release on the Cabal website, but advertise somewhere.

    • For unofficial pre-release, include the ghcup commands for the 3 major artifacts (hosted on Gitlab).
    • For official ones, submit a patch for the ghcup metadata channel (example). In the announcement, remind people how to get the prerelease channel set up for ghcup.

  • If it's a pre-release of 3.8, make the version 3.8.0.date

    • On the Cabal downloads site or GitHub release page, name the release 3.8.1.0-rc1
    • The full release is going to be 3.8.1.0
    • The pre-release changelog is going to be included in the full-release changelog
    • Wait 1 month for the full release or another rc as needed, unless there's a reason to hurry up (in which case an unofficial pre-release makes more sense)

All below this line is outdated, but should be scavenged for creating a new release checklist above this line.

The last content of https://github.com/haskell/cabal/wiki/Making-a-release

Pre-work

You will need to to

  • have SFTP access to www-origin.haskell.org (ask [email protected]),
  • be in the cabal group on that server (ask [email protected]), and
  • be listed as a maintainer of the Cabal and cabal-install packages on Hackage (ask one of the existing maintainers).

Also ensure that you have the following setup:

  • A GPG key - for signing the release tags. Check with gpg --list-keys
  • GHC and the profiling libraries installed
# Check out the major branch you're making a release based on (you might need -b if it doesn't exist).
git checkout X.YY

# Make sure you have all the changes.
git pull

# Repeat for every commit in master that should be part of this release.
git cherry-pick -x <commit on master>

Make a release of Cabal, the library

# The version you're releasing.
export VERSION=X.YY.M.P

# Check what's new since the previous release of Cabal.
git log Cabal-v<previous release>.. -- Cabal/

# Update the changelog by summarizing the above commit log (feel free to
# editorialize).
cd Cabal
$EDITOR changelog

# Bump the version number (depending on if this is a major, minor, or patch release).
$EDITOR Cabal.cabal
$EDITOR Makefile

# Commit the changes.
git commit -am "Bump Cabal version number to v${VERSION}"

# Make sure that the library builds and that the tests pass.
cabal sandbox init
cabal clean
cabal install --enable-tests --only-dependencies
cabal configure --enable-tests
cabal build
cabal test

# Push to the build bot (https://travis-ci.org/haskell/cabal) and wait for
# the results. This exercises additional GHC versions and can take a couple of hours.
git push

# Update the Makefile for the release (KIND=cabal-latest, SSH_USER=<you>).
# These changes shouldn't be committed and can be reverted once the release is done.
$EDITOR Makefile

# Make the release (includes uploading the tarball to haskell.org/cabal).
make release
# TODO(rthomas) The `-latest` symlinks still need to be updated, this is a bit of a pain with sftp.

# Upload the tarball to Hackage as well.
cabal upload dist/Cabal-${VERSION}.tar.gz

# Tag the release - you will need to unlock your GPG key to sign this (you should be prompted)
git tag -a -s -m "Cabal ${VERSION}" Cabal-v${VERSION}
git push && git push --tags

# Update the website. Change the links in download.html to point to the latest release.
# ssh haskell.org
# $EDITOR /home/web/haskell.org/cabal/download.html
# TODO(rthomas) - This needs to be downloaded with sftp, edited and then uploaded again...

Make a release of cabal-install, the executable

Make sure the corresponding Cabal library version (e.g. the one you built above) is installed before you start.

# The version you're releasing.
export VERSION=X.YY.M.P

# Check what's new since the previous release of Cabal.
git log cabal-install-v<previous release>.. -- cabal-install/

# Update the changelog by summarizing the above commit log (feel free to
# editorialize).
cd cabal-install
$EDITOR changelog

# Bump the version number (depending on if this is a major, minor, or patch
# release). You might need to bump the dependency on Cabal as well, if this
# cabal-install release should accompany a new major Cabal release.
$EDITOR cabal-install.cabal
$EDITOR bootstrap.sh

# Commit the changes.
git commit -am "Bump cabal-install version number to v${VERSION}"

# Make sure that the executable builds.
cabal update
cabal sandbox init
#cabal sandbox add-source ../Cabal  # If you're building against in-tree Cabal
cabal clean
cabal install --only-dependencies
cabal configure
cabal build

# Make sure bootstrapping works.
cabal sdist
cp dist/cabal-install-${VERSION}.tar.gz /tmp
pushd /tmp
tar zxf cabal-install-${VERSION}.tar.gz 
cd cabal-install-${VERSION}
sh bootstrap.sh 
popd

# Push to the build bot (https://travis-ci.org/haskell/cabal) and wait for
# the results. This exercises additional GHC versions.
git push

# Upload to Hackage.
cabal upload dist/cabal-install-${VERSION}.tar.gz

# Tag the release.
git tag -a -s -m "cabal-install ${VERSION}" cabal-install-v${VERSION}
git push && git push --tags

# Update the website. Change the links in download.html to point to the latest release.
sftp [email protected]
cd cabal/release
mkdir cabal-install-$(VERSION)
cd cabal-install-$(VERSION)
lcd dist
put cabal-install-$(VERSION).tar.gz
# TODO(rthomas) the cabal-install-latest symlink still needs to be updated

# Update the download.html page
# Update the GitHub releases page
Clone this wiki locally