test: Fix hot restart test under --runs_per_test#2609
test: Fix hot restart test under --runs_per_test#2609dnoe merged 3 commits intoenvoyproxy:masterfrom
Conversation
When using --runs_per_test with concurrency the individual invocations of the hot restart test may collide. Bazel provides an environment variable when using --runs_per_test that allows us to set --base_id in each parallel invocation to a guaranteed unique value so this doesn't happen. Signed-off-by: Dan Noé <dpn@google.com>
| if [[ -z "${TEST_RANDOM_SEED}" ]]; then | ||
| BASE_ID=1 | ||
| else | ||
| BASE_ID="${TEST_RANDOM_SEED}" |
There was a problem hiding this comment.
Arguably this still has the issue that if you run two distinct bazel test on the same machine, with different run IDs, you will have potential conflict. I think it's fine for now, this is a useful improvement over status quo.
There was a problem hiding this comment.
Doesn't Bazel serialize separate build or test invocations made on the same machine in the same code base? When I try it in a separate window it tells me it's waiting for the first command to complete.
There was a problem hiding this comment.
I think it's per-bazel-daemon. On an interactive system, that's generally 1 per user (I think). On a build machine, it's probably 1 per container.
There was a problem hiding this comment.
It's per source directory actually (I know because I run multiple concurrently in different Envoy trees :).
|
Amusingly, looks like it has failed in CI on the real flake in hotrestart test in async grpc client |
Envoy sets up SHM even when not starting up fully, so even cases where the binary is not started with a full config want --base-id. Otherwise we can get "file exists" when running many tests in parallel. Signed-off-by: Dan Noé <dpn@google.com>
|
I think all the cases where the envoy binary is launched from the hot restart test are now properly configured with base-id. |
|
Please merge master to pick up #2613, this will unblock CI. |
* Update proxy to envoy-wasm@9dc356, 12/17 * ReSync to latest envoyproxy/envoy-wasm@9cf22b8 * Fix build * Lint
Automated changes by [create-pull-request](https://github.com/peter-evans/create-pull-request) GitHub action Signed-off-by: JP Simard <jp@jpsim.com>
Automated changes by [create-pull-request](https://github.com/peter-evans/create-pull-request) GitHub action Signed-off-by: JP Simard <jp@jpsim.com>
4 participants
Description:
When using --runs_per_test with concurrency the individual invocations
of the hot restart test may collide. Bazel provides an environment
variable when using --runs_per_test that allows us to set --base_id in
each parallel invocation to a guaranteed unique value so this doesn't
happen.
Risk Level: Low
Testing: bazel test //test/integration:hotrestart_test --runs_per_test=100
Signed-off-by: Dan Noé dpn@google.com