[CI] Run TimeZoneInfo tests in parallel. #4161
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR is intended to give us greater control over when we run our extended regression test suite and to parallelize TimeZoneInfo tests.
Create RunAllTests variable
Creates a new
RunAllTestsvariable that defaults tofalse. This variable is used as aconditionto trigger running our extended regression test suite.This variable is set to
truein the following ways:masterpipeline sets the value totrueso all commits tomasterrun the full suite.mono-mono-2019-12) to run the full suite, as there is a much higher chance of breakage for these commits.Variablesoption will allow you to set it totruefor that run. (The variable shows up with the default value so you don't have to remember its name.)Put extended suites behind RunAllTests variable
The following test suites are behind this new variable:
Integrated Regression - MacOSIntegrated Regression - WindowsTimeZoneInfo - MacOSThe effective changes are:
We have determined that these suites are unlikely to break for normal day-to-day changes, so by default they will now only run on
mastercommits and Mono bump PRs.As stated above, if you feel your change may be risky you can manually kick off a full test in the Pipelines UI.
Updates the regression test stage introduced in commit 47e00d7 to only
run when the
mac_buildstage completes successfully, as it dependson artifacts produced by that stage.
Parallelize TimeZoneInfo tests
The
TimeZoneInfotest suite has been split across 3 build nodes to reduce each run below the target of 1 hour. (Previously this step took over 2 hours.)Testing
Note: Steps not related to this change were canceled or skipped.
RunAllTests = truerun: https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=3407576mono-named branch run: https://devdiv.visualstudio.com/DevDiv/_build/results?buildId=3407804