Skip to content

[fix] refine release instructions #8381

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
villebro merged 7 commits into from
Oct 23, 2019
Merged

[fix] refine release instructions #8381

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
fd8bd85
Separate RC from VERSION
villebro Oct 11, 2019
7376a48
Fix pypi_push.sh and refine instructions
villebro Oct 13, 2019
dae2d1f
Add SUPERSET_ prefix to env variables
villebro Oct 19, 2019
6ce6a59
Finalize release instructions
villebro Oct 22, 2019
e3282ba
Change one-off to First Time Only
villebro Oct 22, 2019
8990398
Add tagging of final version
villebro Oct 22, 2019
40ce743
Convert some remaining env variables and add a check step to pypi deploy
villebro Oct 23, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
113 changes: 85 additions & 28 deletions RELEASING/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ synchronous communications, we tend to use the `#apache-releases` channel
on the Superset Slack. People crafting releases and those interested in
partaking in the process should join the channel.

## Release setup
## Release setup (First Time Only)

First you need to setup a few things. This is a one-off and doesn't
need to be done at every release.
Expand All @@ -53,56 +53,114 @@ need to be done at every release.
svn commit -m "Add PGP keys of new Superset committer"
```

## Crafting tarball and signatures
## Setting up the release environment (do every time)

As the vote process takes a minimum of 72h (community vote) + 72h (IPMC) vote,
often stretching over several weeks calendar time if votes don't pass, chances are
the same terminal session won't be used for crafting the release candidate and the
final release. Therefore, it's a good idea to do the following every time you
work on a new phase of the release process to make sure you aren't releasing
the wrong files/using wrong names:

```bash
# Set SUPERSET_VERSION to the release being prepared, e.g. 0.34.1.
export SUPERSET_VERSION=XX.YY.ZZ
# Set RC to the release candindate number. Replacing QQ below with 1
# indicates rc1 i.e. first vote on version above (0.34.1rc1)
export SUPERSET_RC=QQ
```

Then you can generate other derived environment variables that are used
throughout the release process:

Now let's craft a source release
```bash
# Assuming these commands are executed from the root of the repo
export REPO_DIR=$(pwd)
# Set VERSION to the release being prepared (rc1 for first vote on version)
export VERSION=0.34.1rc1
export RELEASE=apache-superset-incubating-${VERSION}
export RELEASE_TARBALL=${RELEASE}-source.tar.gz
export SUPERSET_VERSION_RC=${SUPERSET_VERSION}rc${SUPERSET_RC}
export SUPERSET_RELEASE=apache-superset-incubating-${SUPERSET_VERSION}
export SUPERSET_RELEASE_RC=apache-superset-incubating-${SUPERSET_VERSION_RC}
export SUPERSET_RELEASE_TARBALL=${SUPERSET_RELEASE}-source.tar.gz
export SUPERSET_RELEASE_RC_TARBALL=${SUPERSET_RELEASE_RC}-source.tar.gz
```

## Preparing the release candidate

The first step of preparing an Apache Release is packaging a release candidate
to be voted on. Start by going to the root of the repo and making sure the
prerequisites are in order:

```bash
# Go to the root directory of the repo, e.g. `~/src/incubator-superset`
cd ~/src/incubator-superset/
export SUPERSET_REPO_DIR=$(pwd)
# make sure you're on the correct branch (e.g. 0.34)
git branch
```

Make sure the version number under `superset/assets/package.json` corresponds
to `SUPERSET_VERSION` above (`0.34.1` in example above), and has been committed to the
branch.

```bash
grep ${SUPERSET_VERSION} superset/assets/package.json
```

If nothing shows up, either the version isn't correctly set in `package.json`,
or the environment variable is misconfigured.

### Crafting tarball and signatures

Now let's craft a source release
```bash
# Let's create a git tag
git tag -f ${VERSION}
git tag -f ${SUPERSET_VERSION_RC}

# Create the target folder
mkdir -p ~/svn/superset_dev/${VERSION}/
mkdir -p ~/svn/superset_dev/${SUPERSET_VERSION_RC}/
git archive \
--format=tar.gz ${VERSION} \
--prefix="${RELEASE}/" \
-o ~/svn/superset_dev/${VERSION}/${RELEASE_TARBALL}
--format=tar.gz ${SUPERSET_VERSION_RC} \
--prefix="${SUPERSET_RELEASE_RC}/" \
-o ~/svn/superset_dev/${SUPERSET_VERSION_RC}/${SUPERSET_RELEASE_RC_TARBALL}

cd ~/svn/superset_dev/${VERSION}/
${REPO_DIR}/scripts/sign.sh ${RELEASE}-source.tar.gz
cd ~/svn/superset_dev/${SUPERSET_VERSION_RC}/
${SUPERSET_REPO_DIR}/scripts/sign.sh ${SUPERSET_RELEASE_RC}-source.tar.gz
```

## Shipping to SVN
### Shipping to SVN

Now let's ship this RC into svn's dev folder

```bash
cd ~/svn/superset_dev/
svn add ${VERSION}
svn commit -m "${VERSION}"
svn add ${SUPERSET_VERSION_RC}
svn commit -m "Release ${SUPERSET_VERSION_RC}"
```

Now you're ready to start the VOTE thread.

## Validating a release
### Validating a release

https://www.apache.org/info/verification.html

## Publishing a successful release

Upon a successful vote, you'll have to copy the folder into the non-"dev/"
folder.
Upon a successful vote (community AND IPMC), you'll have to copy the folder
into the non-"dev/" folder.
```bash
cp -r ~/svn/superset_dev/${VERSION}/ ~/svn/superset/${VERSION}/
cp -r ~/svn/superset_dev/${SUPERSET_VERSION_RC}/ ~/svn/superset/${SUPERSET_VERSION}/
cd ~/svn/superset/
svn add ${VERSION}
svn commit -m "${VERSION}"
# Rename the RC (0.34.1rc1) to the actual version being released (0.34.1)
for f in ${SUPERSET_VERSION}/*; do mv "$f" "${f/${SUPERSET_VERSION_RC}/${SUPERSET_VERSION}}"; done
svn add ${SUPERSET_VERSION}
svn commit -m "Release ${SUPERSET_VERSION}"
```

Then tag the final release:
```bash
# Go to the root directory of the repo, e.g. `~/src/incubator-superset`
cd ~/src/incubator-superset/
# make sure you're on the correct branch (e.g. 0.34)
git branch
# Create the release tag
git tag -f ${SUPERSET_VERSION}
Copy link
Member

@mistercrunch mistercrunch Oct 23, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh, gotta push that to the apache remote here too

# push the branch
git push apache {branch}
# push the tag
git push apache {tag}

In the one-off setup, we may want to have an instruction to create the remote for apache:
git remote add apache git@github.com:apache/incubator-superset.git

Copy link
Member Author

@villebro villebro Oct 23, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps leave those to #8432 ? This is already pretty crowded, perhaps best to get this out of the way now if there's nothing that can't wait until the next PR?

```

Now you can announce the release on the mailing list, make sure to use the
Expand All @@ -120,11 +178,10 @@ section for the new release.
To make a working build given a tarball
```bash
# Building a docker from a tarball
VERSION=0.33.0rc2 && \
docker build -t apache-superset:$VERSION -f Dockerfile.from_tarball . --build-arg VERSION=$VERSION
docker build -t apache-superset:${SUPERSET_VERSION_RC} -f Dockerfile.from_tarball . --build-arg VERSION=${SUPERSET_VERSION_RC}

# testing the resulting docker
docker run -p 5001:8088 apache-superset:$VERSION
docker run -p 5001:8088 apache-superset:${SUPERSET_VERSION_RC}
# you should be able to access localhost:5001 on your browser
# login using admin/admin
```
Expand Down
10 changes: 6 additions & 4 deletions pypi_push.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,13 @@
# See the License for the specific language governing permissions and
# limitations under the License.
#
# first bump up package.json manually, commit and tag
# Make sure you've followed the instructions on `RELEASING/README.md`
# and are on the correct branch
cd ${SUPERSET_REPO_DIR}
git branch
rm superset/assets/dist/*
cd superset/assets/
npm run build
npm ci && npm run build
cd ../..
python setup.py sdist
echo "RUN: twine upload dist/superset-{VERSION}.tar.gz"

echo "RUN: twine upload dist/apache-superset-{SUPERSET_VERSION}.tar.gz"