Skip to content

[1.18] Windows ci add bazel args (#16643) #17068

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
lizan merged 1 commit into from
Jun 25, 2021
Merged

[1.18] Windows ci add bazel args (#16643) #17068

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 3 additions & 6 deletions bazel/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,9 +23,6 @@ powershell Invoke-WebRequest https://github.com/bazelbuild/bazelisk/releases/lat
set PATH=%PATH%;%USERPROFILE%\bazel
```

If you're building from an revision of Envoy prior to August 2019, which doesn't contains a `.bazelversion` file, run `ci/run_envoy_docker.sh "bazel version"`
to find the right version of Bazel and set the version to `USE_BAZEL_VERSION` environment variable to build.

## Production environments

To build Envoy with Bazel in a production environment, where the [Envoy
Expand Down Expand Up @@ -156,9 +153,9 @@ for how to update or override dependencies.
and Bazel rules which follow POSIX python conventions. Add `pip.exe` to the PATH and install the `wheel`
package.
```cmd
mklink %USERPROFILE%\Python38\python3.exe %USERPROFILE%\Python38\python.exe
set PATH=%PATH%;%USERPROFILE%\Python38
set PATH=%PATH%;%USERPROFILE%\Python38\Scripts
mklink %USERPROFILE%\Python39\python3.exe %USERPROFILE%\Python39\python.exe
set PATH=%PATH%;%USERPROFILE%\Python39
set PATH=%PATH%;%USERPROFILE%\Python39\Scripts
pip install wheel
```

Expand Down
12 changes: 7 additions & 5 deletions ci/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ Currently there are three build images for Linux and one for Windows:
* `envoyproxy/envoy-build` — alias to `envoyproxy/envoy-build-ubuntu`.
* `envoyproxy/envoy-build-ubuntu` — based on Ubuntu 18.04 (Bionic) with GCC 9 and Clang 10 compiler.
* `envoyproxy/envoy-build-centos` — based on CentOS 7 with GCC 9 and Clang 10 compiler, this image is experimental and not well tested.
* `envoyproxy/envoy-build-windows2019` — based on Windows 2019 LTS with VS 2019 Build Tools.
* `envoyproxy/envoy-build-windows2019` — based on Windows ltsc2019 with VS 2019 Build Tools, as well as LLVM.

The source for these images is located in the [envoyproxy/envoy-build-tools](https://github.com/envoyproxy/envoy-build-tools)
repository.
Expand Down Expand Up @@ -148,15 +148,15 @@ An example basic invocation to build the Envoy static binary and run tests is:
./ci/run_envoy_docker.sh './ci/windows_ci_steps.sh'
```

You can modify `./ci/windows_ci_steps.sh` to modify `bazel` arguments, tests to run, etc. as well
You can pass additional command line arguments to `./ci/windows_ci_steps.sh` to list specific `bazel` arguments and build/test targets.
as set environment variables to adjust your container build environment as described above.

The Envoy binary can be found in `C:\Windows\Temp\envoy-docker-build\envoy\source\exe` on the Docker host. You
The Envoy binary can be found in `${TEMP}\envoy-docker-build\envoy\source\exe` on the Docker host. You
can control this by setting `ENVOY_DOCKER_BUILD_DIR` in the environment, e.g. to
generate the binary in `C:\Users\foo\build\envoy\source\exe` you can run:

```bash
ENVOY_DOCKER_BUILD_DIR="C:\Users\foo\build" ./ci/run_envoy_docker.sh './ci/do_ci.sh bazel.dev'
ENVOY_DOCKER_BUILD_DIR="C:\Users\foo\build" ./ci/run_envoy_docker.sh './ci/windows_ci_steps.sh'
```

Note the quotations around the `ENVOY_DOCKER_BUILD_DIR` value to preserve the backslashes in the
Expand All @@ -168,7 +168,9 @@ If you would like to run an interactive session to keep the build container runn
./ci/run_envoy_docker.sh 'bash'
```

From an interactive session, you can invoke `bazel` manually or use the `./ci/windows_ci_steps.sh` script to build and run tests.
From an interactive session, you can invoke `bazel` directly, or use the `./ci/windows_ci_steps.sh` script to build and run tests.
Bazel will look for .bazelrc in the `${HOME}` path, which is mapped to the persistent path `${TEMP}\envoy-docker-build\` on the
Docker host.

# Testing changes to the build image as a developer

Expand Down
59 changes: 45 additions & 14 deletions ci/windows_ci_steps.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,24 +67,55 @@ if [[ "${BAZEL_BUILD_EXTRA_OPTIONS[*]}" =~ "clang-cl" ]]; then
FAIL_GROUP=clang_cl
fi

# Pre-Validate updates of all dependency libraries in bazel/foreign_cc and bazel/external
# and complete envoy-static build
bazel "${BAZEL_STARTUP_OPTIONS[@]}" build "${BAZEL_BUILD_OPTIONS[@]}" //bazel/... //source/exe:envoy-static --build_tag_filters=-skip_on_windows

# Copy binary to delivery directory
cp -f bazel-bin/source/exe/envoy-static.exe "${ENVOY_DELIVERY_DIR}/envoy.exe"
# Optional arguments include //source/exe:envoy-static to build,
# //test/... to test all with flake handling and test tag filters
# (these are the default), either one or the other, or a list of
# explicit tests or patterns which does not perform tag exclusions,
# unless given as additional argument. (If we explicitly ask, we
# are determined to fight a broken test, whether it is tagged
# skip/fail on windows or not.)

if [[ $1 == "//source/exe:envoy-static" ]]; then
BUILD_ENVOY_STATIC=1
shift
TEST_TARGETS=$*
elif [[ $# -gt 0 ]]; then
BUILD_ENVOY_STATIC=0
TEST_TARGETS=$*
else
BUILD_ENVOY_STATIC=1
TEST_TARGETS='//test/...'
fi

# Copy for azp, creating a tar archive
tar czf "${ENVOY_BUILD_DIR}"/envoy_binary.tar.gz -C "${ENVOY_DELIVERY_DIR}" envoy.exe
# Complete envoy-static build
if [[ $BUILD_ENVOY_STATIC -eq 1 ]]; then
bazel "${BAZEL_STARTUP_OPTIONS[@]}" build "${BAZEL_BUILD_OPTIONS[@]}" //source/exe:envoy-static

# Test invocations of known-working tests on Windows
bazel "${BAZEL_STARTUP_OPTIONS[@]}" test "${BAZEL_BUILD_OPTIONS[@]}" //test/... --test_tag_filters=-skip_on_windows,-fails_on_${FAIL_GROUP} --build_tests_only
# Copy binary to delivery directory
cp -f bazel-bin/source/exe/envoy-static.exe "${ENVOY_DELIVERY_DIR}/envoy.exe"

echo "running flaky test reporting script"
"${ENVOY_SRCDIR}"/ci/flaky_test/run_process_xml.sh "$CI_TARGET"
# Copy for azp, creating a tar archive
tar czf "${ENVOY_BUILD_DIR}"/envoy_binary.tar.gz -C "${ENVOY_DELIVERY_DIR}" envoy.exe
fi

# Build tests that are known flaky or failing to ensure no compilation regressions
bazel "${BAZEL_STARTUP_OPTIONS[@]}" build "${BAZEL_BUILD_OPTIONS[@]}" //test/... --test_tag_filters=-skip_on_windows,fails_on_${FAIL_GROUP} --build_tests_only
# Test invocations of known-working tests on Windows
if [[ $TEST_TARGETS == "//test/..." ]]; then
bazel "${BAZEL_STARTUP_OPTIONS[@]}" test "${BAZEL_BUILD_OPTIONS[@]}" $TEST_TARGETS --test_tag_filters=-skip_on_windows,-fails_on_${FAIL_GROUP} --build_tests_only

echo "running flaky test reporting script"
"${ENVOY_SRCDIR}"/ci/flaky_test/run_process_xml.sh "$CI_TARGET"

# Build tests that are known flaky or failing to ensure no compilation regressions
bazel "${BAZEL_STARTUP_OPTIONS[@]}" build "${BAZEL_BUILD_OPTIONS[@]}" //test/... --test_tag_filters=fails_on_${FAIL_GROUP} --build_tests_only

if [[ $BUILD_ENVOY_STATIC -eq 1 ]]; then
# Validate introduction or updates of any dependency libraries in bazel/foreign_cc and bazel/external
# not triggered by envoy-static or //test/... targets and not deliberately tagged skip_on_windows
bazel "${BAZEL_STARTUP_OPTIONS[@]}" build "${BAZEL_BUILD_OPTIONS[@]}" //bazel/... --build_tag_filters=-skip_on_windows
fi
elif [[ -n "$TEST_TARGETS" ]]; then
bazel "${BAZEL_STARTUP_OPTIONS[@]}" test "${BAZEL_BUILD_OPTIONS[@]}" $TEST_TARGETS --build_tests_only
fi

# Summarize known unbuildable or inapplicable tests (example)
# bazel "${BAZEL_STARTUP_OPTIONS[@]}" query 'kind(".*test rule", attr("tags", "skip_on_windows", //test/...))' 2>/dev/null | sort
2 changes: 2 additions & 0 deletions test/extensions/http/header_formatters/preserve_case/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,8 @@ envoy_extension_cc_test(
"preserve_case_formatter_integration_test.cc",
],
extension_name = "envoy.http.stateful_header_formatters.preserve_case",
# Broken until bazel 5.0.0 fix to shorten resulting paths for SymInitialize() failure
tags = ["skip_on_windows"],
deps = [
"//source/extensions/http/header_formatters/preserve_case:preserve_case_formatter",
"//test/integration:http_integration_lib",
Expand Down
2 changes: 2 additions & 0 deletions test/extensions/transport_sockets/tls/cert_validator/spiffe/BUILD
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,8 @@ envoy_extension_cc_test(
"//test/extensions/transport_sockets/tls/test_data:certs",
],
extension_name = "envoy.tls.cert_validator.spiffe",
# Broken until bazel 5.0.0 fix to shorten resulting paths for SymInitialize() failure
tags = ["skip_on_windows"],
deps = [
"//source/extensions/transport_sockets/tls/cert_validator/spiffe:config",
"//test/integration:http_integration_lib",
Expand Down