Deflake and speed up parts of ExecutorStepTest
#224
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
calling setLabels on an agent will not persist the node. in older versions of Jenkins the tests would be less flaky as adding any Node would cause all labels to be re-evaluated, so when creating a few agents and adding labels in a loop the last one created would at least deterministically ensure that all previous agents labels where correct. However since 2.332 (jenkinsci/jenkins#5882) only labels part of a node added or removed would be updated, and when creating the agents they where created without labels, which where added later. This caused tests to be flaky depending on when the periodic `trimLabels` was called (or at least on other timing related things) THis was discovered by enabling a loggerrule for hudson.model.queue and observing that the builds would timeout as not all the agents would have the expected nodes. e.g. ``` 12.023 [id=141] FINEST hudson.model.Queue#maintain: JobOffer[ jenkinsci#1] rejected part of demo jenkinsci#1: ?Jenkins? doesn?t have label ?foo? 12.023 [id=141] FINEST hudson.model.Queue#maintain: JobOffer[slave0 #0] is a potential candidate for task part of demo jenkinsci#1 12.024 [id=141] FINEST hudson.model.Queue#maintain: JobOffer[slave2 #0] rejected part of demo jenkinsci#1: ?slave2? doesn?t have label ?foo? 12.024 [id=141] FINEST hudson.model.Queue#maintain: JobOffer[slave1 #0] rejected part of demo jenkinsci#1: ?slave1? doesn?t have label ?foo? 12.024 [id=141] FINEST hudson.model.Queue#maintain: JobOffer[ #0] rejected part of demo jenkinsci#1: ?Jenkins? doesn?t have label ?foo? 12.024 [id=141] FINEST hudson.model.Queue#maintain: JobOffer[slave3 #0] rejected part of demo jenkinsci#1: ?slave3? doesn?t have label ?foo? ``` from `reuseNodesWithSameLabelsInParallelStages` Additionally creating agents and waiting for them to come oneline is slow. A pipeline will start and then wait for the node to be available, so we can do other things whilst the agent is connecting. For the case where we need a number of agents connected before we start to run the pipeline, we now create iall the agents before waiting for them to connect.
jtnord
force-pushed
the
deflake-ExecutorStepTest
branch
from
5f726ea to
47e6c43
Compare
|
(could switch label to |
These are currently the most flaky of all, so |
jglick
requested changes
May 11, 2022
| @@ -139,10 +140,8 @@ public class ExecutorStepTest { | |||
| */ | |||
| @Test public void buildShellScriptOnSlave() throws Throwable { | |||
| sessions.then(r -> { | |||
| DumbSlave s = r.createOnlineSlave(); | |||
| s.setLabelString("remote quick"); | |||
| DumbSlave s = r.createSlave("remote quick", null); | |||
There was a problem hiding this comment.
Comment on lines
771
to
773
| // Note: for Jenkins versions > 2.65, the number of agents must be increased to 5. | ||
| // This is due to changes in the Load Balancer (See JENKINS-60563). | ||
| int totalAgents = Jenkins.getVersion().isNewerThan(new VersionNumber("2.265")) ? 5 : 3; |
There was a problem hiding this comment.
BTW the comment does not match the version number string, and both are older than the baseline…could be cleaned up (but best left to another PR I suppose).
src/test/java/org/jenkinsci/plugins/workflow/support/steps/ExecutorStepTest.java
Outdated
Show resolved
Hide resolved
jglick
changed the title
ExecutorStepTest
…cutorStepTest.java Co-authored-by: Jesse Glick <[email protected]>
jglick
approved these changes
May 11, 2022
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
calling
setLabelson an agent will not persist the node.in older versions of Jenkins the tests would be less flaky as adding any
Node would cause all labels to be re-evaluated, so when creating a few
agents and adding labels in a loop the last one created would at least
deterministically ensure that all previous agents labels where correct.
However since 2.332 (jenkinsci/jenkins#5882)
only labels part of a node added or removed would be updated, and when
creating the agents they where created without labels, which where added
later.
This caused tests to be flaky depending on when the periodic
trimLabelswas called (or at least on other timing related things)THis was discovered by enabling a loggerrule for hudson.model.queue and
observing that the builds would timeout as not all the agents would have
the expected nodes.
e.g.
from
reuseNodesWithSameLabelsInParallelStagesAdditionally creating agents and waiting for them to come oneline is
slow. A pipeline will start and then wait for the node to be available,
so we can do other things whilst the agent is connecting.
For the case where we need a number of agents connected before we start
to run the pipeline, we now create iall the agents before waiting for them to connect.