ci: update do_ci.sh targets and docs to the bazel test matrix.

.travis.yml

@@ -7,11 +7,10 @@ install:
 matrix:
   fast_finish: true
 env:
-  - TEST_TYPE=debug
+  - TEST_TYPE=bazel.release
+  - TEST_TYPE=bazel.asan
   - TEST_TYPE=bazel.coverage
-  - TEST_TYPE=bazel.debug
-  - TEST_TYPE=normal
-  - TEST_TYPE=asan
+  - TEST_TYPE=check_format
   - TEST_TYPE=docs
 script: ./ci/ci_steps.sh
 

ci/README.md

@@ -13,71 +13,28 @@ binary built from the latest tip of master that passed tests.
 
 ## Alpine envoy image
 
-Minimal images based on alpine Linux allow for quicker deployment of Envoy. Two alpine based images are built,
+Minimal images based on Alpine Linux allow for quicker deployment of Envoy. Two Alpine based images are built,
 one with an Envoy binary with debug (`lyft/envoy-alpine-debug`) symbols and one stripped of them (`lyft/envoy-alpine`).
 Both images are pushed with two different tags: `<hash>` and `latest`. Parallel to the Ubuntu images above, `<hash>` corresponds to the
 master commit at which the binary was compiled, and `latest` corresponds to a binary built from the latest tip of master that passed tests.
 
-# Building a debug image
-An example basic invocation to build a debug image and run all tests is:
+# Building and running tests as a developer
 
-```bash
-docker pull lyft/envoy-build:latest && docker run -t -i -u $(id -u):$(id -g) -v <BUILD_DIR>:/build -v <SOURCE_DIR>:/source lyft/envoy-build:latest /bin/bash -c "cd /source && ci/do_ci.sh debug"
-```
-
-On OSX using the command below may work better. Unlike on Linux, users are not
-synced between the container and OS. Additionally, the bind mount will not
-create artifacts with the same ownership as in the container ([read more about
-osxfs][osxfs]).
+An example basic invocation to build the Envoy static binary (using the Bazel `fastbuild` type) is:
 
 ```bash
-docker pull lyft/envoy-build:latest && docker run -t -i -u root:root -v <BUILD_DIR>:/build -v <SOURCE_DIR>:/source lyft/envoy-build:latest /bin/bash -c "cd /source && ci/do_ci.sh debug"
+./ci/run_envoy_docker.sh bazel.fastbuild
 ```
 
-This bind mounts `<SOURCE_DIR>`, which allows for changes on the local
-filesystem to be consumed and outputs build artifacts in `<SOURCE_DIR>/build`.
-The static Envoy binary can be found in `<SOURCE_DIR>/build_debug/source/exe/envoy`.
+The output artifacts can be found in `/build/envoy/bazel-bin/source/exe/envoy-static` in the
+container.
 
 The `do_ci.sh` targets are:
 
-* `asan` &mdash; build and run tests with [AddressSanitizer](https://github.com/google/sanitizers/wiki/AddressSanitizer).
-* `coverage` &mdash; build and run tests, generating coverage information in `<SOURCE_DIR>/build_coverage/coverage.html`.
-* `debug` &mdash; build debug binary and run tests.
-* `bazel.coverage` &mdash; build and run tests with Bazel, generating coverage information in `<SOURCE_DIR>/generated/coverage/coverage.html`.
-* `bazel.debug` &mdash; build debug binary and run tests with Bazel.
-* `fix_format`&mdash; run `clang-format` 3.6 on entire source tree.
-* `normal` &mdash; build unstripped optimized binary and run tests .
-* `server_only` &mdash; build stripped optimized binary only.
-
-A convenient shell function to define is:
-
-```bash
-run_envoy_docker() { docker pull lyft/envoy-build:latest && docker run -t -i -u $(id -u):$(id -g) -v /tmp/envoy-docker-build:/build -v $PWD:/source lyft/envoy-build:latest /bin/bash -c "cd /source && $*";}
-```
-
-Or on OSX.
-
-```bash
-run_envoy_docker() { docker pull lyft/envoy-build:latest && docker run -t -i -u root:root -v /tmp/envoy-docker-build:/build -v $PWD:/source lyft/envoy-build:latest /bin/bash -c "cd /source && $*";}
-```
-
-This then allows for a simple invocation of `run_envoy_docker './ci/do_ci.sh debug'` from the
-Envoy source tree.
-
-## Advanced developer features
-
-* Any parameters following the `do_ci.sh` target are passed in as command-line
-  arguments to the `envoy-test` binary during unit test execution. This allows
-  for [GTest](https://github.com/google/googletest) flags to be passed, e.g.
-  `run_envoy_docker './ci/do_ci.sh debug --gtest_filter="*Dns*"'`.
-
-* A `UNIT_TEST_ONLY` environment variable is available to control test execution to limit testing to
-  just unit tests, e.g. `run_envoy_docker 'UNIT_TEST_ONLY=1 ./ci/do_ci.sh debug --gtest_filter="*Dns*"'`.
-
-* A `RUN_TEST_UNDER` environment variable is available to specify an executable to run the tests
-  under. For example, to run a subset of tests under `gdb`: `run_envoy_docker 'RUN_TEST_UNDER="gdb --args" UNIT_TEST_ONLY=1 ./ci/do_ci.sh debug --gtest_filter="*Dns*"'`.
-
-* A `SKIP_CHECK_FORMAT` environment variable is available to skip `clang-format` checks while developing locally, e.g. `run_envoy_docker 'SKIP_CHECK_FORMAT=1 ./ci/do_ci.sh debug'`.
-
-
-[osxfs]: https://docs.docker.com/docker-for-mac/osxfs/
+* `bazel.asan` &mdash; build and run tests under `-c dbg --config=asan`.
+* `bazel.fastbuild` &mdash; build Envoy static binary under `-c fastbuild`.
+* `bazel.fastbuild.test` &mdash; build and run tests under `-c fastbuild`.
+* `bazel.release` &mdash; build Envoy static binary and run tests under `-c opt`.
+* `bazel.coverage` &mdash; build and run tests under `-c dbg`, generating coverage information in `<SOURCE_DIR>/generated/coverage/coverage.html`.
+* `check_format`&mdash; run `clang-format` 3.6 and `buildifier` on entire source tree.
+* `fix_format`&mdash; run and enforce `clang-format` 3.6 and `buildifier` on entire source tree.

ci/build_setup.sh

@@ -1,7 +1,6 @@
 #!/bin/bash
 
-# Configure environment variables, generate makefiles and switch to build
-# directory in preparation for an invocation of the generated build makefiles.
+# Configure environment variables for Bazel build and test.
 
 set -e
 
@@ -14,108 +13,64 @@ NUM_CPUS=`grep -c ^processor /proc/cpuinfo`
 
 export ENVOY_SRCDIR=/source
 
-if [[ "$1" == bazel* ]]
-then
-  # Create a fake home. Python site libs tries to do getpwuid(3) if we don't and the CI
-  # Docker image gets confused as it has no passwd entry when running non-root
-  # unless we do this.
-  FAKE_HOME=/tmp/fake_home
-  mkdir -p "${FAKE_HOME}"
-  export HOME="${FAKE_HOME}"
-  export PYTHONUSERBASE="${FAKE_HOME}"
+# Create a fake home. Python site libs tries to do getpwuid(3) if we don't and the CI
+# Docker image gets confused as it has no passwd entry when running non-root
+# unless we do this.
+FAKE_HOME=/tmp/fake_home
+mkdir -p "${FAKE_HOME}"
+export HOME="${FAKE_HOME}"
+export PYTHONUSERBASE="${FAKE_HOME}"
 
-  export BUILD_DIR=/build
-  # Make sure that "docker run" has a -v bind mount for /build, since cmake
-  # users will only have a bind mount for /source.
-  if [[ ! -d "${BUILD_DIR}" ]]
-  then
-    echo "${BUILD_DIR} mount missing - did you forget -v <something>:${BUILD_DIR}?"
-    exit 1
-  fi
-  export ENVOY_CONSUMER_SRCDIR="${BUILD_DIR}/envoy-consumer"
+export BUILD_DIR=/build
+# Make sure that "docker run" has a -v bind mount for /build, since cmake
+# users will only have a bind mount for /source.
+if [[ ! -d "${BUILD_DIR}" ]]
+then
+  echo "${BUILD_DIR} mount missing - did you forget -v <something>:${BUILD_DIR}?"
+  exit 1
+fi
+export ENVOY_CONSUMER_SRCDIR="${BUILD_DIR}/envoy-consumer"
 
-  # Make sure that /source doesn't contain /build on the underlying host
-  # filesystem, including via hard links or symlinks. We can get into weird
-  # loops with Bazel symlinking and gcovr's path traversal if this is true, so
-  # best to keep /source and /build in distinct directories on the host
-  # filesystem.
-  SENTINEL="${BUILD_DIR}"/bazel.sentinel
-  touch "${SENTINEL}"
-  if [[ -n "$(find -L "${ENVOY_SRCDIR}" -name "$(basename "${SENTINEL}")")" ]]
-  then
-    rm -f "${SENTINEL}"
-    echo "/source mount must not contain /build mount"
-    exit 1
-  fi
+# Make sure that /source doesn't contain /build on the underlying host
+# filesystem, including via hard links or symlinks. We can get into weird
+# loops with Bazel symlinking and gcovr's path traversal if this is true, so
+# best to keep /source and /build in distinct directories on the host
+# filesystem.
+SENTINEL="${BUILD_DIR}"/bazel.sentinel
+touch "${SENTINEL}"
+if [[ -n "$(find -L "${ENVOY_SRCDIR}" -name "$(basename "${SENTINEL}")")" ]]
+then
   rm -f "${SENTINEL}"
+  echo "/source mount must not contain /build mount"
+  exit 1
+fi
+rm -f "${SENTINEL}"
 
-  # Environment setup.
-  export USER=bazel
-  export TEST_TMPDIR=/build/tmp
-  export BAZEL="bazel"
-  # Not sandboxing, since non-privileged Docker can't do nested namespaces.
-  BAZEL_OPTIONS="--package_path %workspace%:/source"
-  export BAZEL_QUERY_OPTIONS="${BAZEL_OPTIONS}"
-  export BAZEL_BUILD_OPTIONS="--strategy=Genrule=standalone --spawn_strategy=standalone \
-    --verbose_failures ${BAZEL_OPTIONS} --action_env=HOME --action_env=PYTHONUSERBASE"
-  export BAZEL_TEST_OPTIONS="${BAZEL_BUILD_OPTIONS} --test_env=HOME --test_env=PYTHONUSERBASE"
-  [[ "${BAZEL_EXPUNGE}" == "1" ]] && "${BAZEL}" clean --expunge
-  ln -sf /thirdparty "${ENVOY_SRCDIR}"/ci/prebuilt
-  ln -sf /thirdparty_build "${ENVOY_SRCDIR}"/ci/prebuilt
-
-  # Setup Envoy consuming project.
-  if [[ ! -a "${ENVOY_CONSUMER_SRCDIR}" ]]
-  then
-    # TODO(htuch): Update to non-htuch when https://github.com/lyft/envoy/issues/404 is sorted.
-    git clone https://github.com/htuch/envoy-consumer.git "${ENVOY_CONSUMER_SRCDIR}"
-  fi
-  cp -f "${ENVOY_SRCDIR}"/ci/WORKSPACE.consumer "${ENVOY_CONSUMER_SRCDIR}"/WORKSPACE
-  # This is the hash on https://github.com/htuch/envoy-consumer.git we pin to.
-  (cd "${ENVOY_CONSUMER_SRCDIR}" && git checkout 94e11fa753a1e787c82cccaec642eda5e5b61ed8)
-
-  # Also setup some space for building Envoy standalone.
-  export ENVOY_BUILD_DIR="${BUILD_DIR}"/envoy
-  mkdir -p "${ENVOY_BUILD_DIR}"
-  cp -f "${ENVOY_SRCDIR}"/ci/WORKSPACE "${ENVOY_BUILD_DIR}"
-else
-  # TODO(htuch): Remove everything below this comment when we turn off cmake.
-  if [[ "$1" == "coverage" ]]; then
-    EXTRA_CMAKE_FLAGS+=" -DENVOY_CODE_COVERAGE:BOOL=ON"
-  elif [[ "$1" == "asan" ]]; then
-    EXTRA_CMAKE_FLAGS+=" -DENVOY_SANITIZE:BOOL=ON -DENVOY_DEBUG:BOOL=OFF"
-  elif [[ "$1" == "debug" ]]; then
-    EXTRA_CMAKE_FLAGS+=" -DENVOY_DEBUG:BOOL=ON"
-  elif [[ "$1" == "server_only" ]]; then
-    EXTRA_CMAKE_FLAGS+=" -DENVOY_DEBUG:BOOL=OFF -DENVOY_STRIP:BOOL=ON"
-  else
-    EXTRA_CMAKE_FLAGS+=" -DENVOY_DEBUG:BOOL=OFF"
-  fi
-
-  mkdir -p build_"$1"
-  cd build_"$1"
-
-  cmake \
-  ${EXTRA_CMAKE_FLAGS} \
-  -DENVOY_COTIRE_MODULE_DIR:FILEPATH=/thirdparty/cotire-cotire-1.7.8/CMake \
-  -DENVOY_GMOCK_INCLUDE_DIR:FILEPATH=/thirdparty_build/include \
-  -DENVOY_GPERFTOOLS_INCLUDE_DIR:FILEPATH=/thirdparty_build/include \
-  -DENVOY_GTEST_INCLUDE_DIR:FILEPATH=/thirdparty_build/include \
-  -DENVOY_HTTP_PARSER_INCLUDE_DIR:FILEPATH=/thirdparty_build/include \
-  -DENVOY_LIBEVENT_INCLUDE_DIR:FILEPATH=/thirdparty_build/include \
-  -DENVOY_CARES_INCLUDE_DIR:FILEPATH=/thirdparty_build/include \
-  -DENVOY_NGHTTP2_INCLUDE_DIR:FILEPATH=/thirdparty_build/include \
-  -DENVOY_SPDLOG_INCLUDE_DIR:FILEPATH=/thirdparty/spdlog-0.11.0/include \
-  -DENVOY_TCLAP_INCLUDE_DIR:FILEPATH=/thirdparty/tclap-1.2.1/include \
-  -DENVOY_OPENSSL_INCLUDE_DIR:FILEPATH=/thirdparty_build/include \
-  -DENVOY_LIGHTSTEP_TRACER_INCLUDE_DIR:FILEPATH=/thirdparty_build/include \
-  -DENVOY_PROTOBUF_INCLUDE_DIR:FILEPATH=/thirdparty_build/include \
-  -DENVOY_PROTOBUF_PROTOC:FILEPATH=/thirdparty_build/bin/protoc \
-  -DENVOY_GCOVR:FILEPATH=/thirdparty/gcovr-3.3/scripts/gcovr \
-  -DENVOY_RAPIDJSON_INCLUDE_DIR:FILEPATH=/thirdparty/rapidjson-1.1.0/include \
-  -DENVOY_GCOVR_EXTRA_ARGS:STRING="-e test/* -e build/*" \
-  -DENVOY_EXE_EXTRA_LINKER_FLAGS:STRING=-L/thirdparty_build/lib \
-  -DENVOY_TEST_EXTRA_LINKER_FLAGS:STRING=-L/thirdparty_build/lib \
-  ..
+# Environment setup.
+export USER=bazel
+export TEST_TMPDIR=/build/tmp
+export BAZEL="bazel"
+# Not sandboxing, since non-privileged Docker can't do nested namespaces.
+BAZEL_OPTIONS="--package_path %workspace%:/source"
+export BAZEL_QUERY_OPTIONS="${BAZEL_OPTIONS}"
+export BAZEL_BUILD_OPTIONS="--strategy=Genrule=standalone --spawn_strategy=standalone \
+  --verbose_failures ${BAZEL_OPTIONS} --action_env=HOME --action_env=PYTHONUSERBASE"
+export BAZEL_TEST_OPTIONS="${BAZEL_BUILD_OPTIONS} --test_env=HOME --test_env=PYTHONUSERBASE"
+[[ "${BAZEL_EXPUNGE}" == "1" ]] && "${BAZEL}" clean --expunge
+ln -sf /thirdparty "${ENVOY_SRCDIR}"/ci/prebuilt
+ln -sf /thirdparty_build "${ENVOY_SRCDIR}"/ci/prebuilt
 
-  cmake -L || true
+# Setup Envoy consuming project.
+if [[ ! -a "${ENVOY_CONSUMER_SRCDIR}" ]]
+then
+  # TODO(htuch): Update to non-htuch when https://github.com/lyft/envoy/issues/404 is sorted.
+  git clone https://github.com/htuch/envoy-consumer.git "${ENVOY_CONSUMER_SRCDIR}"
 fi
+cp -f "${ENVOY_SRCDIR}"/ci/WORKSPACE.consumer "${ENVOY_CONSUMER_SRCDIR}"/WORKSPACE
+# This is the hash on https://github.com/htuch/envoy-consumer.git we pin to.
+(cd "${ENVOY_CONSUMER_SRCDIR}" && git checkout 94e11fa753a1e787c82cccaec642eda5e5b61ed8)
+
+# Also setup some space for building Envoy standalone.
+export ENVOY_BUILD_DIR="${BUILD_DIR}"/envoy
+mkdir -p "${ENVOY_BUILD_DIR}"
+cp -f "${ENVOY_SRCDIR}"/ci/WORKSPACE "${ENVOY_BUILD_DIR}"

ci/ci_steps.sh

@@ -11,7 +11,7 @@ travis lint .travis.yml --skip-completion-check
 if [ $TEST_TYPE == "docs" ]
 then
   echo "docs build..."
-  make docs
+  ./docs/build.sh
   ./docs/publish.sh
   exit 0
 else

ci/do_ci.sh

@@ -7,15 +7,37 @@ set -e
 . "$(dirname "$0")"/build_setup.sh
 echo "building using ${NUM_CPUS} CPUs"
 
-if [[ "$1" == "bazel.debug" ]]; then
-  echo "bazel debug build with tests..."
+if [[ "$1" == "bazel.release" ]]; then
+  echo "bazel release build with tests..."
   cd "${ENVOY_CONSUMER_SRCDIR}"
   echo "Building..."
-  bazel build ${BAZEL_BUILD_OPTIONS} @envoy//source/exe:envoy-static.stamped
+  bazel --batch build ${BAZEL_BUILD_OPTIONS} -c opt @envoy//source/exe:envoy-static.stripped.stamped
   echo "Testing..."
-  bazel test ${BAZEL_TEST_OPTIONS} --test_output=all \
+  bazel --batch test ${BAZEL_TEST_OPTIONS} -c opt --test_output=all \
     --cache_test_results=no @envoy//test/... //:echo2_integration_test
   exit 0
+elif [[ "$1" == "bazel.asan" ]]; then
+  echo "bazel ASAN debug build with tests..."
+  cd "${ENVOY_CONSUMER_SRCDIR}"
+  echo "Building and testing..."
+  # TODO(htuch): This should switch to using clang when available.
+  bazel --batch test ${BAZEL_TEST_OPTIONS} -c dbg --config=asan --test_output=all \
+    --cache_test_results=no @envoy//test/... //:echo2_integration_test
+  exit 0
+elif [[ "$1" == "bazel.fastbuild" ]]; then
+  # This doesn't go into CI but is available for developer convenience.
+  echo "bazel fastbuild of envoy-static..."
+  cd "${ENVOY_BUILD_DIR}"
+  echo "Building..."
+  bazel build ${BAZEL_BUILD_OPTIONS} -c fastbuild //source/exe:envoy-static
+  exit 0
+elif [[ "$1" == "bazel.fastbuild.test" ]]; then
+  # This doesn't go into CI but is available for developer convenience.
+  echo "bazel test with fastbuild..."
+  cd "${ENVOY_BUILD_DIR}"
+  echo "Building and testing..."
+  bazel test ${BAZEL_BUILD_OPTIONS} -c fastbuild //test/...
+  exit 0
 elif [[ "$1" == "bazel.coverage" ]]; then
   echo "bazel coverage build with tests..."
   export GCOVR="/thirdparty/gcovr-3.3/scripts/gcovr"
@@ -32,6 +54,7 @@ elif [[ "$1" == "bazel.coverage" ]]; then
   # some Bazel created symlinks to the source directory in its output
   # directory. Wow.
   cd "${ENVOY_BUILD_DIR}"
+  export BAZEL_TEST_OPTIONS="${BAZEL_TEST_OPTIONS} -c dbg"
   SRCDIR="${GCOVR_DIR}" "${ENVOY_SRCDIR}"/test/run_envoy_bazel_coverage.sh
   exit 0
 elif [[ "$1" == "fix_format" ]]; then
@@ -44,25 +67,7 @@ elif [[ "$1" == "check_format" ]]; then
   cd "${ENVOY_SRCDIR}"
   ./tools/check_format.py check
   exit 0
-elif [[ "$1" == "coverage" ]]; then
-  echo "coverage build with tests..."
-  TEST_TARGET="envoy.check-coverage"
-elif [[ "$1" == "asan" ]]; then
-  echo "asan build with tests..."
-  TEST_TARGET="envoy.check"
-elif [[ "$1" == "debug" ]]; then
-  echo "debug build with tests..."
-  TEST_TARGET="envoy.check"
-elif [[ "$1" == "server_only" ]]; then
-  echo "normal build server only..."
-  TEST_TARGET="envoy"
 else
-  echo "normal build with tests..."
-  TEST_TARGET="envoy.check"
+  echo "Invalid do_ci.sh target, see ci/README.md for valid targets."
+  exit 1
 fi
-
-shift
-export EXTRA_TEST_ARGS="$@"
-
-[[ "${SKIP_CHECK_FORMAT}" == "1" ]] || make check_format
-make -j${NUM_CPUS} ${TEST_TARGET}

ci/run_envoy_docker.sh

@@ -3,9 +3,9 @@
 set -e
 
 [[ -z "${IMAGE_ID}" ]] && IMAGE_ID="latest"
+[[ -z "${ENVOY_DOCKER_BUILD_DIR}" ]] && ENVOY_DOCKER_BUILD_DIR=/tmp/envoy-docker-build
 
-BUILD_DIR=/tmp/envoy-docker-build
-mkdir -p "${BUILD_DIR}"
+mkdir -p "${ENVOY_DOCKER_BUILD_DIR}"
 docker pull lyft/envoy-build:"${IMAGE_ID}"
-docker run -t -i -u $(id -u):$(id -g) -v "${BUILD_DIR}":/build \
+docker run -t -i -u $(id -u):$(id -g) -v "${ENVOY_DOCKER_BUILD_DIR}":/build \
   -v "$PWD":/source lyft/envoy-build:"${IMAGE_ID}" /bin/bash -c "cd source && $*"