Closed
Conversation
Running the e2e bats test sequentially took 15m on my laptop. We should run them all in parallel such that we don't have to wait so long on CI.
basvandijk
force-pushed
the
basvandijk/parallel-e2e-bats-tests
branch
from
651d173 to
52e773a
Compare
mergify bot
pushed a commit
that referenced
this pull request
Sep 30, 2020
…tstrap to pick random webserver ports (#920) I'm going to merge these commits into #877 So far this PR does a couple things - fixes the fact that ipv6 addresses could not be provided to `dfx start` - fixes that both `dfx bootstrap` and `dfx start` could not handle the user providing port `0` i.e. telling dfx to pick a random port for the webserver - writes the port used by the webserver to `.dfx/webserver-port` (similar to `.dfx/pid`) so that it can be queried by dfx (or by the bats test) the second and third bullet will enable the bats tests to run in parallel in darwin (w/o network sandbox) since we'll tell dfx start or dfx bootstrap to use port `0`, the kernel will dynamically allocate a port, and the bats test can read `.dfx/webserver-port`
Contributor
Author
|
This has been merged into #920. |
dfinity-bot
added a commit
that referenced
this pull request
Apr 29, 2021
## Changelog for advisory-db: Branch: master Commits: [rustsec/advisory-db@5b3dcb83...db03320a](rustsec/advisory-db@5b3dcb8...db03320) * [`7086fa22`](rustsec/advisory-db@7086fa2) Report rkyv vulnerability (issue rkyv[RustSec/advisory-db#113](http://r.duckduckgo.com/l/?uddg=https://github.com/RustSec/advisory-db/issues/113)) ([RustSec/advisory-db#877](http://r.duckduckgo.com/l/?uddg=https://github.com/RustSec/advisory-db/issues/877)) * [`db03320a`](rustsec/advisory-db@db03320) Assigned RUSTSEC-2021-0054 to rkyv ([RustSec/advisory-db#878](http://r.duckduckgo.com/l/?uddg=https://github.com/RustSec/advisory-db/issues/878))
mergify bot
pushed a commit
that referenced
this pull request
Apr 29, 2021
## Changelog for advisory-db: Branch: master Commits: [rustsec/advisory-db@5b3dcb83...db03320a](rustsec/advisory-db@5b3dcb8...db03320) * [`7086fa22`](rustsec/advisory-db@7086fa2) Report rkyv vulnerability (issue rkyv[RustSec/advisory-db#113](http://r.duckduckgo.com/l/?uddg=https://github.com/RustSec/advisory-db/issues/113)) ([RustSec/advisory-db#877](http://r.duckduckgo.com/l/?uddg=https://github.com/RustSec/advisory-db/issues/877)) * [`db03320a`](rustsec/advisory-db@db03320) Assigned RUSTSEC-2021-0054 to rkyv ([RustSec/advisory-db#878](http://r.duckduckgo.com/l/?uddg=https://github.com/RustSec/advisory-db/issues/878))
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Running the e2e bats test sequentially took 15m on my laptop. We should run them all in parallel such that we don't have to wait so long on CI. In the following CI step execution plot you can see that the tests now run in parallel and take only ~3m:
We do get a lot of "Address already in use" errors on darwin. Probably because the network is not sandboxed on darwin causing the many parallel tests to interfere with each other. These are essentially flaky tests because ideally the test should cope when an address is already in use. It should then just pick a new address.