[v1.x] Backport staggered CI builds (#17999 & #18119) (#18141)

* For mxnet-validation pipeline, require sanity build to complete successfully before running other build pipelines. (#17999) * Refactor staggered builds - create new full build pipeline that runs sanity check first, then starts all other builds. * Move list of build jobs to top of file for clarity. Preserve whole job path in case we use nested folders in the future. Co-authored-by: Joe Evans <[email protected]> * If sanity build is not found, wait until Jenkins recognizes it. (#18119) * If sanity build is not found, wait until Jenkins recognizes it. * Also add a timeout of 30m for sanity build to run and complete, so we don't get stuck in a loop. Co-authored-by: Joe Evans <[email protected]> Co-authored-by: Joe Evans <[email protected]>
apache · Apr 24, 2020 · 0e7dd91 · 0e7dd91
1 parent f765e8a
commit 0e7dd91
Showing 1 changed file with 61 additions and 0 deletions.
diff --git a/ci/jenkins/Jenkinsfile_full b/ci/jenkins/Jenkinsfile_full
@@ -0,0 +1,61 @@
+// -*- mode: groovy -*-
+
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+//
+// Jenkins pipeline
+// See documents at https://jenkins.io/doc/book/pipeline/jenkinsfile/
+
+// timeout in minutes
+def max_time = 30
+
+def buildJobs = [
+    'centos-cpu',
+    'centos-gpu',
+    'clang',
+    'edge',
+    'miscellaneous',
+    'unix-cpu',
+    'unix-gpu',
+    'website',
+    'windows-cpu',
+    'windows-gpu'
+]
+
+
+stage("full-build") {
+    timeout(time: max_time, unit: 'MINUTES') {
+        // get the base path by removing build and branch portions
+        def jobPath = JOB_NAME.split('/')
+        def pipelineName = jobPath[0..jobPath.size()-3].join('/')
+        def sanityDone = false
+        while (!sanityDone) {
+            try {
+                println("Attempting to run sanity build...")
+                build job: pipelineName + "/sanity/" + BRANCH_NAME, wait: true
+                sanityDone = true
+            } catch (hudson.AbortException e) {
+                println("Job doesn't yet exist, waiting for Jenkins to find job..")
+                sleep(5)
+            }
+        }
+        buildJobs.each { subJob ->
+            build job: pipelineName + "/" + subJob + "/" + BRANCH_NAME, wait: false
+        }
+    }
+}
+