Skip to content

Build: Save build artifact and publish release #2633

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
mattklein123 merged 11 commits into from
Mar 14, 2018
Merged

Build: Save build artifact and publish release #2633

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
a21b226
Build: Save build artifact and publish release
taion809 Feb 16, 2018
7fd5126
Added save_artifacts to save artifacts on ci release
taion809 Feb 16, 2018
3c28ffd
wrap mv in a file test
taion809 Feb 16, 2018
73023f3
fixed github_release to validate and use envvar
taion809 Feb 16, 2018
442f364
moved publishing to separate script
taion809 Feb 18, 2018
7ff0f07
Fix publish script path
taion809 Feb 18, 2018
65d34d9
exit 1 when environment variables are unset
taion809 Feb 21, 2018
9ee83f6
bugfix: ensure we only run when CIRCLE_TAG is populated
taion809 Feb 28, 2018
4e3a215
remove unnecessary if block
taion809 Feb 28, 2018
6327141
move tag check to beginning for early exit
taion809 Feb 28, 2018
557e8ae
only attach a file to an existing release
taion809 Feb 28, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .circleci/config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,12 @@ jobs:
steps:
- checkout
- run: ci/do_circle_ci.sh bazel.release
- run: ci/do_circle_ci.sh github_release

@bndw bndw Feb 28, 2018

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this will be run on every repository event, resulting in a release created for every merge?

We've historically handled this by adding a check in the script to only publish/release for tag events. Here's an example

@taion809 taion809 Feb 28, 2018

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this should be fixed, great catch!

- setup_remote_docker
- run: ci/docker_push.sh
- run: ci/docker_tag.sh
- store_artifacts:
path: /tmp/envoy-dist
asan:
docker:
- image: *envoy-build-image
Expand Down
51 changes: 51 additions & 0 deletions ci/do_ci.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,11 @@ function bazel_release_binary_build() {
# TODO(mattklein123): Replace this with caching and a different job which creates images.
echo "Copying release binary for image build..."
mkdir -p "${ENVOY_SRCDIR}"/build_release
mkdir -p /tmp/envoy-dist
cp -f "${ENVOY_DELIVERY_DIR}"/envoy "${ENVOY_SRCDIR}"/build_release
mkdir -p "${ENVOY_SRCDIR}"/build_release_stripped
strip "${ENVOY_DELIVERY_DIR}"/envoy -o "${ENVOY_SRCDIR}"/build_release_stripped/envoy
cp "${ENVOY_SRCDIR}"/build_release_stripped/envoy /tmp/envoy-dist/envoy
}

function bazel_debug_binary_build() {
Expand All @@ -36,6 +38,20 @@ function bazel_debug_binary_build() {
"${ENVOY_DELIVERY_DIR}"/envoy-debug
}

function publish_github_release() {
TAG=$(git describe --abbrev=0 --tags)

wget https://github.com/aktau/github-release/releases/download/v0.7.2/linux-amd64-github-release.tar.bz2 -O /tmp/ghrelease.tar.bz2
tar -xvjpf /tmp/ghrelease.tar.bz2 -C /tmp
cp /tmp/bin/linux/amd64/github-release /usr/local/bin/ghrelease
chmod +x /usr/local/bin/ghrelease

if [[ -n "${TAG:-}" ]]; then
ghrelease release --tag "${TAG:-}" --name "${TAG:-}"
ghrelease upload --tag "${TAG:-}" --name "envoy-linux-amd64" --file "${ENVOY_SRCDIR}/build_release_stripped/envoy"
fi
}

if [[ "$1" == "bazel.release" ]]; then
# The release build step still runs during tag events. Avoid rebuilding for no reason.
# TODO(mattklein123): Consider moving this into its own "build".
Expand Down Expand Up @@ -167,6 +183,41 @@ elif [[ "$1" == "check_format" ]]; then
elif [[ "$1" == "docs" ]]; then
docs/publish.sh
exit 0
elif [[ "$1" == "github_release" ]]; then
if [[ ! -f "${ENVOY_SRCDIR}/build_release_stripped/envoy" ]]; then

@htuch htuch Feb 18, 2018

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you move this into a new script ci/publish_github_release.sh? There's a fair bit of logic here that would be easier to understand in isolation. In the new script, can you also add a comment at the top explicitly warning not to set set -x debug, since we can get dangerous leakage of GH creds via CI logs if we don't take this step. This hit us previously in Docker CI image push, see https://github.com/envoyproxy/envoy/blob/master/ci/build_container/docker_push.sh#L3.

echo "could not locate envoy binary at path: ${ENVOY_SRCDIR}/build_release_stripped/envoy"

# TODO(taion809): discuss whether or not failing to publish to github warrents failing the build itself

@htuch htuch Feb 18, 2018

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think misconfiguration that is this bad, i.e. we're in a completely unexpected environment, should warrant failure via exit 1.

exit 0
fi

if [[ -z "${GITHUB_TOKEN:-}" ]]; then
echo "environment variable GITHUB_TOKEN unset; cannot continue with publishing."

# TODO(taion809): discuss whether or not failing to publish to github warrents failing the build itself
exit 0
fi

if [[ -z "${GITHUB_USER:-}" ]]; then
echo "environment variable GITHUB_USERNAME unset; cannot continue with publishing."

# TODO(taion809): discuss whether or not failing to publish to github warrents failing the build itself
exit 0
fi

if [[ -z "${GITHUB_REPO:-}" ]]; then
echo "environment variable GITHUB_USERNAME unset; cannot continue with publishing."

# TODO(taion809): discuss whether or not failing to publish to github warrents failing the build itself
exit 0
fi

if [[ -n "${CIRCLE_TAG:-}" ]]; then
echo "skipping tag events"
exit 0
fi

publish_github_release
else
echo "Invalid do_ci.sh target, see ci/README.md for valid targets."
exit 1
Expand Down
8 changes: 5 additions & 3 deletions ci/do_circle_ci.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,11 @@

set -e

# bazel uses jgit internally and the default circle-ci .gitconfig says to
# convert https://github.com to ssh://git@github.com, which jgit does not support.
mv ~/.gitconfig ~/.gitconfig_save
if [[ -f ~/.gitconfig ]]; then
# bazel uses jgit internally and the default circle-ci .gitconfig says to
# convert https://github.com to ssh://git@github.com, which jgit does not support.
mv ~/.gitconfig ~/.gitconfig_save
fi

export ENVOY_SRCDIR="$(pwd)"

Expand Down