Skip to content

How to release GATK4

Jump to bottom
Louis Bergelson edited this page Oct 24, 2024 · 31 revisions

One-time setup tasks

In order to be able to release GATK4, you first need to perform the following one-time setup tasks:

  • Set up your sonatype account

    • If you don't already have a sonatype account, create one by going to https://issues.sonatype.org/secure/Signup!default.jspa.

    • Check whether your sonatype account already has Deploy and UI Upload permissions for repo target org.broadinstitute by going to https://oss.sonatype.org/#stagingProfiles while logged in.

    • If you don't have Deploy and UI Upload permissions for org.broadinstitute, you need to request them. Ask someone who already has deployment permission to open a ticket on your behalf at https://issues.sonatype.org requesting permissions for you, and comment on the ticket yourself from your sonatype account once it's opened.

  • Set up your signing keys for publishing to sonatype

    • (These instructions are adapted from https://central.sonatype.org/publish/requirements/gpg/)

    • Install GPG 1.x (gpg1) via homebrew or your system's package manager. GPG 2.x does not work with our build system.

    • Run gpg1 --gen-key to generate a new keypair. Select the default values when asked for the kind and the size of the keys, and be sure to specify a passphrase for the keypair.

    • Run gpg1 --list-keys and gpg1 --list-secret-keys. The output should look something like this:

      $ gpg1 --list-keys
/Users/YOUR_USERNAME/.gnupg/pubring.gpg
---------------------------------
pub   4096R/4C1B2A36 2016-05-17
uid                  YOUR NAME <YOUR@EMAIL>
sub   4096R/26A37141 2016-05-17

    • Take note of the path to your secret keyring file (eg., /Users/YOUR_USERNAME/.gnupg/secring.gpg) and the public key id (4C1B2A36 in this case).

    • Publish your public key by running gpg1 --keyserver keyserver.ubuntu.com --send-keys YOUR_PUBLIC_KEY_ID, replacing YOUR_PUBLIC_KEY_ID with the key ID for your public key noted above (4C1B2A36 in the example).

      • If this fails with "Server indicated a failure" or "Network is unreachable", try using the IP address of the keyserver instead.
      • If this still fails, try keys.openpgp.org as the keyserver.

  • Set up your sonatype user token

    • Create a sonatype user token by following the instructions in the sonatype docs
    • Save the token username and password somewhere secure. It will go into your gradle properties file in the next step.

  • Set up your credentials in gradle

    • Open ~/.gradle/gradle.properties in a text editor (creating it if necessary).

    • Add the following lines:

      signing.keyId=ID_FOR_YOUR_PUBLIC_KEY
signing.password=PASSPHRASE_FOR_YOUR_KEYPAIR
signing.secretKeyRingFile=PATH_TO_YOUR_SECRET_KEYRING_FILE
sonatypeUsername=YOUR_SONATYPE_TOKEN_USERNAME
sonatypePassword=YOUR_SONATYPE_TOKEN_PASSWORD

    • Save the file.

  • Set up docker and your dockerhub account

    • Install docker from docker.com, and start the docker daemon.

    • Test that you can pull and run the latest GATK4 docker image:

      docker pull broadinstitute/gatk:latest
docker run -it broadinstitute/gatk:latest

    • If you don't already have a dockerhub account, create one by going to hub.docker.com

    • Make sure that you have push access to thehttps://hub.docker.com/r/broadinstitute/gatk/ repository on dockerhub, as well as the us.gcr.io/broad-gatk/gatk repository on GCR. Ask to join the gatk-release google group, which is used to manage these permissions (you may need to verify that you have been granted access).

    • Enable "Experimental Features" in your Docker settings (Docker -> Preferences -> Daemon).

  • Set up your gcloud credentials

    • Follow the instructions in the GATK README to install the Google Cloud SDK and authenticate with gcloud.

    • Make sure your gcloud project is set to broad-dsde-methods by running the command:

      gcloud config set project broad-dsde-methods

    • Run gcloud auth configure-docker to allow docker to push to Google Cloud repositories

Releasing GATK4

After you've performed the one-time setup steps above, here's what you need to do each time you want to release GATK4:

  • Before releasing

    • Check with other GATK developers to confirm that it's a good time to release. In particular:

      • Tools and arguments not ready for production use should be marked as @BetaFeature

      • If a developer has added unstable functionality to a previously-stable tool, their new functionality should be isolated into a separate code path that's off by default and activated using an argument marked as experimental.

    • Make sure that all tests are passing on master in travis!

  • Writing release notes

    • Write release notes in advance and save them in a text editor, otherwise you'll introduce a significant delay in the middle of the release process.

    • Use a markdown-formatted bulleted list, and include a bullet point for each new feature or bug fix in the release, with a brief summary of the major changes at the top. Consult with other GATK developers if necessary to understand new features/patches. Try to include links to the relevant github issue(s) for each item.

    • You can get a list of the commits since the last release by going to the tag for the previous release in https://github.com/broadinstitute/gatk/releases and clicking the X commits to master since this release link. This can form the basis for your release notes.

    • A good way to quickly auto-generate a skeleton for markdown-formatted release notes is by running this command from your GATK4 clone with the latest master checked out:

      git log --pretty=oneline PREVIOUS_VERSION..HEAD | grep -v "Merge pull request" | awk '{ $1=""; print "* " $0; }'

      Replacing PREVIOUS_VERSION with the tag for the previous release

      The following does the same but looks up the last tag automatically.

      git log --pretty=oneline --no-merges $(git describe --abbrev=0)..HEAD | awk '{ $1=""; print "* " $0; }'

      This version of the command also groups the commits by author:

      git log --no-merges '--pretty=format:%an %h %s' $(git describe --abbrev=0)..HEAD | sort | awk '{ print "* " $0; }'

      You'll have to tweak the output of these commands to produce the final release notes, as not all commit messages will be sufficiently descriptive, and you'll need to add links to the appropriate github issues.

      This template may be helpful in formatting the notes.

      **Download release:** [gatk-VERSION.zip](https://github.com/broadinstitute/gatk/releases/download/VERSION/gatk-VERSION.zip)
**Docker image:** [https://hub.docker.com/r/broadinstitute/gatk/](https://hub.docker.com/r/broadinstitute/gatk/)

**Highlights of the VERSION release:**
--------------------------------------
* 

**Full list of changes:**
-------------------------

* **New Tools**
    * 

* **HaplotypeCaller**
    * 
    
* **Mutect2**
    * 
    
* **Joint Calling**
    * 

* **GenomicsDB**
    * 
    
* **Funcotator**
    * 
    
* **Mitochondrial pipeline** 
    * 

* **CNV Calling**
    * 
    
* **SV Calling**
    *

* **Notable Enhancements**
    *

* **Bug Fixes**
    *

* **Miscellaneous Changes**
    * 
    
* **Documentation**
    * 
    
* **Dependencies**
    *

  • Preparing your git clone

    • In your git clone of GATK4, save any uncommitted work and run git checkout -f master. Running git diff HEAD should produce no output.

    • Run git pull --ff-only from your master branch to bring it up to date. Check the commit at the top of the git log output and make sure that it matches the most recent commit at https://github.com/broadinstitute/gatk/commits/master.

    • Run ./gradlew clean to clean your build directory.

  • Releasing to github

    • Decide on a version number for the new GATK4 release (for example, 4.beta.3).

    • Be sure that you still have the latest master branch commit checked out, as instructed above.

    • Tag the release by running git tag -sa VERSION_NUMBER, where VERSION_NUMBER is the version you've chosen for the new release (eg., 4.beta.3). Your text editor will pop up and ask you to write a commit message for the new tag (this can just be the release number for now). After doing so, save and then exit. git describe should now print the new version.

    • Push the new tag to github by running git push origin VERSION_NUMBER, where VERSION_NUMBER is the tag you just created (for example, git push origin 4.beta.3).

    • Run ./gradlew clean bundle -Drelease=true. This will create a zip for the release in build/

    • Go to https://github.com/broadinstitute/gatk/releases, click on the new tag you just pushed, then click "Edit Release".

    • Attach the GATK4 release zip file you just created by clicking on the Attach binaries by dropping them here or selecting them link, and navigating to the zip file in the build/ subdirectory within your GATK4 clone.

    • Paste the release notes you wrote earlier into the text box. Look at the markdown preview and make sure that they look reasonable.

    • Once the zip file has finished uploading to github, click the Publish Release button.

    • Verify that the new release looks ok in https://github.com/broadinstitute/gatk/releases, and has an attached (non-source-code) GATK4 zip file.

  • Releasing to dockerhub

    • Option 1 (do this if possible): build the docker image in the cloud using Google Cloud Build:

      • Confirm that you've set your gcloud project to broad-dsde-methods via the command gcloud config set project broad-dsde-methods

      • From the root of your GATK clone, run:

        bash build_docker_remote.sh -e ${GITHUB_TAG} -r -d ${STAGING_DIR}

        Replace ${GITHUB_TAG} with the tag for your new release that you created above (eg., 4.0.0.0), and ${STAGING_DIR} with the name of the directory that should be used to stage the image. This directory should either not exist or be empty!

      • The script will push the new image to us.gcr.io/broad-dsde-methods/broad-gatk-snapshots/gatk-remote-builds:${YOUR_USERNAME}-${GITHUB_TAG}-${GIT_HASH_FOR_TAG} by default, though this can be changed by running with the -t option

      • Once the image is built you should pull it and test it out locally before releasing it:

        docker pull ${CLOUD_IMAGE_NAME}
docker run -it ${CLOUD_IMAGE_NAME}

        Replacing ${CLOUD_IMAGE_NAME} with the actual image location output by build_docker_remote.sh

      • Once you've verified that the cloud image runs and looks ok, push it to the official release repositories by running:

        bash scripts/docker/release_prebuilt_docker_image.sh "${CLOUD_IMAGE_NAME}" "${GITHUB_TAG}"

        Where ${CLOUD_IMAGE_NAME} is the cloud-built image pulled above, and ${GITHUB_TAG} is the tag for your new release (eg., 4.0.0.0). You may need to run docker login to authenticate before running this script.

    • Option 2 (backup option for when the cloud build doesn't work): build the docker image locally:

      • Ensure that the docker daemon is running, and that you still have the latest master checked out, as instructed above. You'll also need to have done the steps for the github release already.

      • If you've done a docker release in the past, be sure to delete any old docker staging directories(s) left over in your GATK4 clone. You'll have to use sudo to do the deletion.

      • From the root of your GATK clone, run:

        bash build_docker.sh -e ${GITHUB_TAG} -p -u -d ${STAGING_DIR}

        Replace ${GITHUB_TAG} with the tag for your new release that you created above (eg., 4.beta.3), and ${STAGING_DIR} with the name of the directory that should be used to build the image. This directory should either not exist or be empty!

      • When prompted by the script, enter your dockerhub username and password.

      • After the script completes (may take 15-20 minutes depending on your upload speed), go to https://hub.docker.com/r/broadinstitute/gatk/tags/ and confirm that the new image is there.

      • Add a link to the new image in the release notes you wrote on github.

  • Releasing to maven central

    • Be sure that you still have the latest master branch commit checked out, as instructed above. You'll also need to have done the steps for the github release already.

    • Check GATK4's dependencies in build.gradle. If there are any dependencies on a SNAPSHOT of a library, you can't release to maven central, and should skip the remainder of this section.

    • Run ./gradlew clean publish -Drelease=true. This builds the release and uploads it to sonatype's staging area.

    • Go to https://oss.sonatype.org/#stagingRepositories, logging in if necessary. If you don't see anything, click "refresh".

    • Find the release you just uploaded. It will probably be at the bottom with a name like orgbroadinstitute-1027, with your user ID listed as owner.

    • Check the box next to your release, then select "close". Press the refresh button repeatedly until it updates and its status says "closed".

    • Select your release again and click "release". Select the box to "automatically drop" in the pop-up confirmation dialog.

    • Wait ~30-180 minutes for the maven central release to happen. Once it happens, it will show up on http://maven.org. Due to caching on maven's website it's often available by directly navigating to the coordinates sooner than it shows up in search.

  • Post-Release Tasks

    • Publish the GATK WDLs to the gatk-tool-wdls repository by running the following command from the root of your GATK clone:

      bash scripts/publish_gatk_tool_wdls.sh GATK_VERSION

      where GATK_VERSION is the tag for the GATK release you just published

    • After publishing the GATK WDLs, you'll need to go into the my workflows page on Dockstore and manually publish any unpublished WDLs from the gatk-tool-wdls repository. This step might become possible to automate if https://github.com/dockstore/dockstore/issues/3876 is ever implemented.

    • Publish the GATKDocs to the GATK website

      • Ensure that you have the necessary Python dependencies: pip3 install requests && pip3 install markdown2
      • Consult the private GATK secrets document for the correct values of the GATKEMAIL and GATKTOKEN environment variables, and set them in your environment: 
            export GATKEMAIL=<Value from GATK secrets document>
    export GATKTOKEN=<Value from GATK secrets document>
      • With the newly-released version of GATK checked out, build the GATKDocs: 
            ./gradlew clean gatkDoc -Drelease=true
      • Reformat the GATKDocs and publish them to the website using the GATK_Forum_Update_script_v1.py script hosted in the private GATKWebsiteScripts github repo: 
            python3 GATK_Forum_Update_script_v1.py -V VERSION_OF_NEW_RELEASE -P build/docs/gatkdoc
      • Log in to Zendesk on the GATK website, and mark the new docs as "current" via the following procedure:
        • Select "guide" from the dropdown in upper right
        • Click "guide admin" in upper right
        • Then click "arrange content" on the left sidebar
        • Select "tool index"
        • Move the latest version to the top so that it appears first on the website tool version list
        • Then click "Edit Section" to move the "Current" label from the old version to the new version by manually editing the section names.
Clone this wiki locally