Build: Rework integ test setup and shutdown to ensure stop runs when desired #23304
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…desired Gradle's finalizedBy on tasks only ensures one task runs after another, but not immediately after. This is problematic for our integration tests since it allows multiple project's integ test clusters to be simultaneously. While this has not been a problem thus far (gradle 2.13 happened to keep the finalizedBy tasks close enough that no clusters were running in parallel), with gradle 3.3 the task graph generation has changed, and numerous clusters may be running simultaneously, causing memory pressure, and thus generally slower tests, or even failure if the system has a limited amount of memory (eg in a vagrant host). This commit reworks how integ tests are configured. It adds an `integTestCluster` extension to gradle which is equivalent to the current `integTest.cluster` and moves the rest test runner task to `integTestRunner`. The `integTest` task is then just a dummy task, which depends on the cluster runner task, as well as the cluster stop task. This means running `integTest` in one project will both run the rest tests, and shut down the cluster, before running `integTest` in another project.
rjernst
added
:Delivery/Build
Member
Author
|
Note the interesting changes here are in |
nik9000
approved these changes
Feb 22, 2017
rjernst
added a commit
that referenced
this pull request
Feb 22, 2017
…desired (#23304) Gradle's finalizedBy on tasks only ensures one task runs after another, but not immediately after. This is problematic for our integration tests since it allows multiple project's integ test clusters to be simultaneously. While this has not been a problem thus far (gradle 2.13 happened to keep the finalizedBy tasks close enough that no clusters were running in parallel), with gradle 3.3 the task graph generation has changed, and numerous clusters may be running simultaneously, causing memory pressure, and thus generally slower tests, or even failure if the system has a limited amount of memory (eg in a vagrant host). This commit reworks how integ tests are configured. It adds an `integTestCluster` extension to gradle which is equivalent to the current `integTest.cluster` and moves the rest test runner task to `integTestRunner`. The `integTest` task is then just a dummy task, which depends on the cluster runner task, as well as the cluster stop task. This means running `integTest` in one project will both run the rest tests, and shut down the cluster, before running `integTest` in another project.
rjernst
added a commit
that referenced
this pull request
Feb 22, 2017
…desired (#23304) Gradle's finalizedBy on tasks only ensures one task runs after another, but not immediately after. This is problematic for our integration tests since it allows multiple project's integ test clusters to be simultaneously. While this has not been a problem thus far (gradle 2.13 happened to keep the finalizedBy tasks close enough that no clusters were running in parallel), with gradle 3.3 the task graph generation has changed, and numerous clusters may be running simultaneously, causing memory pressure, and thus generally slower tests, or even failure if the system has a limited amount of memory (eg in a vagrant host). This commit reworks how integ tests are configured. It adds an `integTestCluster` extension to gradle which is equivalent to the current `integTest.cluster` and moves the rest test runner task to `integTestRunner`. The `integTest` task is then just a dummy task, which depends on the cluster runner task, as well as the cluster stop task. This means running `integTest` in one project will both run the rest tests, and shut down the cluster, before running `integTest` in another project.
jasontedor
added a commit
to jasontedor/elasticsearch
that referenced
this pull request
Feb 24, 2017
* master: (54 commits) Keep the pipeline handler queue small initially Do not create String instances in 'Strings' methods accepting StringBuilder (elastic#22907) Tests: fix AwsS3ServiceImplTests Remove abstract InternalMetricsAggregation class (elastic#23326) Add BulkRequest support to High Level Rest client (elastic#23312) Wrap getCredentials() in a doPrivileged() block (elastic#23297) Respect promises on pipelined responses Align REST specs for HEAD requests Remove unnecessary result sorting in SearchPhaseController (elastic#23321) Fix SamplerAggregatorTests to have stable and predictable docIds Tests: Ensure multi node integ tests wait on first node Relocate a comment in HttpPipeliningHandler Add comments to HttpPipeliningHandler [TEST] Fix incorrect test cluster name in cluster health doc tests Build: Change location in zip of license and notice inclusion for plugins (elastic#23316) Script: Fix value of `ctx._now` to be current epoch time in milliseconds (elastic#23175) Build: Rework integ test setup and shutdown to ensure stop runs when desired (elastic#23304) Handle long overflow when adding paths' totals Don't set local node on cluster state used for node join validation (elastic#23311) Ensure that releasing listener is called ...
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Gradle's finalizedBy on tasks only ensures one task runs after another,
but not immediately after. This is problematic for our integration tests
since it allows multiple project's integ test clusters to be
simultaneously. While this has not been a problem thus far (gradle 2.13
happened to keep the finalizedBy tasks close enough that no clusters
were running in parallel), with gradle 3.3 the task graph generation has
changed, and numerous clusters may be running simultaneously, causing
memory pressure, and thus generally slower tests, or even failure if the
system has a limited amount of memory (eg in a vagrant host).
This commit reworks how integ tests are configured. It adds an
integTestClusterextension to gradle which is equivalent to the currentintegTest.clusterand moves the rest test runner task tointegTestRunner. TheintegTesttask is then just a dummy task,which depends on the cluster runner task, as well as the cluster stop
task. This means running
integTestin one project will both run therest tests, and shut down the cluster, before running
integTestinanother project.