Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[DevOps] Isolate docker image build & push #367

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

[DevOps] Isolate docker image build & push #367

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
74d6d99
ci: Add separate stages to build docker images, wheels
ChathurindaRanasinghe Jun 30, 2023
f81000c
ci: Update conditions & parameter names
ChathurindaRanasinghe Jun 30, 2023
1865051
ci: Update parameter names
ChathurindaRanasinghe Jun 30, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
37 changes: 35 additions & 2 deletions ci/jenkinsfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,14 +34,16 @@ pipeline {
parameters {
booleanParam(name: 'skipTesting', defaultValue: false, description: 'Skip testing')
text(name: "testTargets", defaultValue: "${ALL_TESTS.keySet().join('\n')}", description: "A select set of tests to run")
booleanParam(name: 'buildWheels', defaultValue: true, description: 'Build Python wheels')
booleanParam(name: 'buildAndPush', defaultValue: true, description: 'Build h2ogpt docker image & push if image does not exists in harbor')
booleanParam(name: 'publish', defaultValue: false, description: 'Upload to HF')
}
options {
ansiColor('xterm')
timestamps()
}
stages {
stage('Build') {
stage('Init') {
agent {
label "linux && docker"
}
Expand All @@ -51,8 +53,22 @@ pipeline {
def commitMsg = sh(returnStdout: true, script: 'git log -1 --pretty=format:"[%an] %s"').trim()
currentBuild.displayName = "${env.BUILD_ID} - [${shortHash}]"
currentBuild.description = "${commitMsg}"
}
}
}

sh "make docker_build"
stage('Build Wheels') {
when {
anyOf {
expression { return params.buildWheels }
}
beforeAgent true
}
agent {
label "linux && docker"
}
steps {
script {
docker.image("harbor.h2o.ai/library/python:3.10").inside("--entrypoint='' --security-opt seccomp=unconfined -e USE_WHEEL=1 -e HOME=${WORKSPACE}") {
sh "make clean dist"
}
Expand All @@ -63,6 +79,23 @@ pipeline {
}
}

stage('Build & Push Docker Image') {
when {
anyOf {
expression { return params.buildAndPush }
}
beforeAgent true
}
agent {
label "linux && docker"
}
steps {
script {
sh "make docker_build"
}
}
}

stage('Tests') {
when {
anyOf {
Expand Down