Skip to content

CI: Move the Build stage from Azure to Jenkins #12198

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Alexey-Rivkin wants to merge 5 commits into
base: master
Choose a base branch
from
Open

CI: Move the Build stage from Azure to Jenkins #12198

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
cbbac7c
CI: Match Azure rows in the Jenkins build job
Alexey-Rivkin Oct 29, 2026
affcce2
CI: Upload only the failed rows' logs to GitHub
Alexey-Rivkin Nov 2, 2026
441679b
CI: Drop the Azure Build stage
Alexey-Rivkin Nov 6, 2026
b8b2988
CI: Remove config left over from the Azure Build stage
Alexey-Rivkin Nov 6, 2026
18d0ea1
CI: Drop the require_ze and CONTAINER row variables
Alexey-Rivkin Nov 6, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 7 additions & 2 deletions .ci/jenkins/Jenkinsfile.github
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,12 +32,17 @@ try {
throw ex
} finally {
if (params.githubData) {
def passed = currentBuild.resultIsBetterOrEqualTo('SUCCESS')
githubHelper.updateCommitStatus(
pipelineOverviewUrl,
"CI completed",
currentBuild.resultIsBetterOrEqualTo('SUCCESS') ? GitHubCommitState.SUCCESS : GitHubCommitState.FAILURE,
passed ? GitHubCommitState.SUCCESS : GitHubCommitState.FAILURE,
env.JOB_BASE_NAME
)
githubHelper.uploadLogs(this, env.JOB_NAME, env.BUILD_NUMBER, null, null)
// Only the failed rows' logs, with internal hostnames hidden.
if (!passed) {
Comment thread
Alexey-Rivkin marked this conversation as resolved.
Resolved
githubHelper.uploadParallelLogs(this, env.JOB_NAME, env.BUILD_NUMBER, null,
[/[\w.-]+\.nvidia\.com/, /[\w.-]+\.mellanox\.com/, /[\w.-]+\.mlnx\b/])
}
}
}
91 changes: 50 additions & 41 deletions .ci/pipeline/build_matrix.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,70 +5,79 @@
job: ucx-build

failFast: false
timeout_minutes: 90
timeout_minutes: 240

kubernetes:
cloud: il-ipp-blossom-prod-nbu-swx-ucx
namespace: nbu-swx-ucx
limits: "{memory: 16Gi, cpu: 8000m}"
requests: "{memory: 16Gi, cpu: 8000m}"

# Modules for builds.sh.
volumes:
- {mountPath: /hpc/local, hostPath: /hpc/local}
Comment thread
Alexey-Rivkin marked this conversation as resolved.
Resolved
Comment thread
Alexey-Rivkin marked this conversation as resolved.
Resolved

runs_on_dockers:
# x86_64
- { name: "ubuntu2204-mofed5.7", url: "harbor.mellanox.com/ucx/x86_64/ubuntu22.04/builder:mofed-5.7-0.2.3.0", arch: x86_64 }
- { name: "ubuntu2404-doca2.9", url: "harbor.mellanox.com/hpcx/x86_64/ubuntu24.04/builder:doca-2.9.0", arch: x86_64 }
- { name: "ubuntu2604-mofed26.04", url: "harbor.mellanox.com/hpcx/x86_64/ubuntu26.04/builder:mofed-26.04-0.7.6.0", arch: x86_64 }
- { name: "ubuntu2610-inbox", url: "harbor.mellanox.com/hpcx/x86_64/ubuntu26.10/builder:inbox", arch: x86_64 }
- { name: "rocky89-mofed24.10", url: "harbor.mellanox.com/hpcx/x86_64/rocky8.9/builder:mofed-24.10-3.2.5.0", arch: x86_64 }
- { name: "rocky96-mofed24.10", url: "harbor.mellanox.com/hpcx/x86_64/rocky9.6/builder:mofed-24.10-3.2.5.0", arch: x86_64 }
- { name: "debian125-doca2.9", url: "harbor.mellanox.com/hpcx/x86_64/debian12.5/builder:doca-2.9.0", arch: x86_64 }
- { name: "debian113-mofed5.8", url: "harbor.mellanox.com/hpcx/x86_64/debian11.3/builder:mofed-5.8-3.0.7.0", arch: x86_64 }
- { name: "debian130-doca3.2", url: "harbor.mellanox.com/hpcx/x86_64/debian13.0/builder:doca-3.2.0", arch: x86_64 }
- { name: "centos10stream-inbox", url: "harbor.mellanox.com/hpcx/x86_64/centos10stream/builder:inbox", arch: x86_64 }
- { name: "fedora44-inbox", url: "harbor.mellanox.com/hpcx/x86_64/fedora44/builder:inbox", arch: x86_64 }
- { name: "ubuntu2204-mofed5.7", url: "harbor.mellanox.com/ucx/x86_64/ubuntu22.04/builder:mofed-5.7-0.2.3.0", arch: x86_64, build_mode: short }
Comment thread
Alexey-Rivkin marked this conversation as resolved.
Resolved
Comment thread
Alexey-Rivkin marked this conversation as resolved.
Resolved
Comment thread
Alexey-Rivkin marked this conversation as resolved.
Resolved
Comment thread
Alexey-Rivkin marked this conversation as resolved.
Resolved
- { name: "ubuntu2404-doca2.9", url: "harbor.mellanox.com/hpcx/x86_64/ubuntu24.04/builder:doca-2.9.0", arch: x86_64, build_mode: long }
- { name: "ubuntu2604-mofed26.04", url: "harbor.mellanox.com/hpcx/x86_64/ubuntu26.04/builder:mofed-26.04-0.7.6.0", arch: x86_64, build_mode: short }
- { name: "ubuntu2610-inbox", url: "harbor.mellanox.com/hpcx/x86_64/ubuntu26.10/builder:inbox", arch: x86_64, build_mode: sanity }
- { name: "rocky89-mofed24.10", url: "harbor.mellanox.com/hpcx/x86_64/rocky8.9/builder:mofed-24.10-3.2.5.0", arch: x86_64, build_mode: sanity }
- { name: "rocky96-mofed24.10", url: "harbor.mellanox.com/hpcx/x86_64/rocky9.6/builder:mofed-24.10-3.2.5.0", arch: x86_64, build_mode: sanity }
- { name: "debian125-doca2.9", url: "harbor.mellanox.com/hpcx/x86_64/debian12.5/builder:doca-2.9.0", arch: x86_64, build_mode: short }
- { name: "debian113-mofed5.8", url: "harbor.mellanox.com/hpcx/x86_64/debian11.3/builder:mofed-5.8-3.0.7.0", arch: x86_64, build_mode: sanity }
- { name: "debian130-doca3.2", url: "harbor.mellanox.com/hpcx/x86_64/debian13.0/builder:doca-3.2.0", arch: x86_64, build_mode: short }
- { name: "centos10stream-inbox", url: "harbor.mellanox.com/hpcx/x86_64/centos10stream/builder:inbox", arch: x86_64, build_mode: sanity }
- { name: "fedora44-inbox", url: "harbor.mellanox.com/hpcx/x86_64/fedora44/builder:inbox", arch: x86_64, build_mode: sanity }
# Azure Linux 4.0 is in public preview; the image tracks the azurelinux-beta channel.
- { name: "azurelinux40-inbox", url: "harbor.mellanox.com/hpcx/x86_64/azurelinux4.0/builder:inbox", arch: x86_64 }
- { name: "rhel82-mofed5.0", url: "harbor.mellanox.com/ucx/x86_64/rhel8.2/builder:mofed-5.0-1.0.0.0", arch: x86_64 }
- { name: "rhel90-mofed5.6", url: "harbor.mellanox.com/ucx/x86_64/rhel9.0/builder:mofed-5.6-0.5.0.0", arch: x86_64 }
- { name: "rhel100-mofed25.07", url: "harbor.mellanox.com/hpcx/x86_64/rhel10.0/builder:mofed-25.07-0.9.7.0", arch: x86_64 }
- { name: "sles15sp7-mofed26.01", url: "harbor.mellanox.com/hpcx/x86_64/sles15sp7/builder:mofed-26.01-0.8.8.0", arch: x86_64 }
- { name: "sles16sp0-mofed26.01", url: "harbor.mellanox.com/hpcx/x86_64/sles16sp0/builder:mofed-26.01-0.8.0.0", arch: x86_64 }
- { name: "openeuler2203sp4-doca3.5", url: "harbor.mellanox.com/hpcx/x86_64/openeuler22.03sp4/builder:doca-3.5.0", arch: x86_64 }
- { name: "openeuler2403sp3-doca3.5", url: "harbor.mellanox.com/hpcx/x86_64/openeuler24.03sp3/builder:doca-3.5.0", arch: x86_64 }
- { name: "ctyunos2507-mofed26.01", url: "harbor.mellanox.com/hpcx/x86_64/ctyunos25.07/builder:mofed-26.01-0.8.0.0", arch: x86_64 }
- { name: "tencentos44-mofed26.04", url: "harbor.mellanox.com/hpcx/x86_64/tencentos4.4/builder:mofed-26.04-0.7.6.0", arch: x86_64 }
- { name: "azurelinux40-inbox", url: "harbor.mellanox.com/hpcx/x86_64/azurelinux4.0/builder:inbox", arch: x86_64, build_mode: sanity }
- { name: "rhel82-mofed5.0", url: "harbor.mellanox.com/ucx/x86_64/rhel8.2/builder:mofed-5.0-1.0.0.0", arch: x86_64, build_mode: short }
- { name: "rhel82-mofed5.0-gcc15.2", url: "harbor.mellanox.com/ucx/x86_64/rhel8.2/builder:mofed-5.0-1.0.0.0", arch: x86_64, build_mode: sanity, GCC_MODULE: "dev/gcc-15.2.0" }
Comment thread
Alexey-Rivkin marked this conversation as resolved.
Resolved
- { name: "rhel90-mofed5.6", url: "harbor.mellanox.com/ucx/x86_64/rhel9.0/builder:mofed-5.6-0.5.0.0", arch: x86_64, build_mode: short }
- { name: "rhel100-mofed25.07", url: "harbor.mellanox.com/hpcx/x86_64/rhel10.0/builder:mofed-25.07-0.9.7.0", arch: x86_64, build_mode: long }
- { name: "sles15sp7-mofed26.01", url: "harbor.mellanox.com/hpcx/x86_64/sles15sp7/builder:mofed-26.01-0.8.8.0", arch: x86_64, build_mode: short }
- { name: "sles16sp0-mofed26.01", url: "harbor.mellanox.com/hpcx/x86_64/sles16sp0/builder:mofed-26.01-0.8.0.0", arch: x86_64, build_mode: short }
- { name: "openeuler2203sp4-doca3.5", url: "harbor.mellanox.com/hpcx/x86_64/openeuler22.03sp4/builder:doca-3.5.0", arch: x86_64, build_mode: short }
- { name: "openeuler2403sp3-doca3.5", url: "harbor.mellanox.com/hpcx/x86_64/openeuler24.03sp3/builder:doca-3.5.0", arch: x86_64, build_mode: short }
- { name: "ctyunos2507-mofed26.01", url: "harbor.mellanox.com/hpcx/x86_64/ctyunos25.07/builder:mofed-26.01-0.8.0.0", arch: x86_64, build_mode: short }
- { name: "tencentos44-mofed26.04", url: "harbor.mellanox.com/hpcx/x86_64/tencentos4.4/builder:mofed-26.04-0.7.6.0", arch: x86_64, build_mode: short }
# 20.1060a is the newest UOS rootfs on urm; DOCA publishes it for x86_64 only.
- { name: "uos201060a-doca3.4", url: "harbor.mellanox.com/hpcx/x86_64/uos20.1060a/builder:doca-3.4.0", arch: x86_64 }
- { name: "ubuntu2204-rocm6.0", url: "harbor.mellanox.com/ucx/x86_64/ubuntu2204:rocm-6.0.0", arch: x86_64 }
- { name: "uos201060a-doca3.4", url: "harbor.mellanox.com/hpcx/x86_64/uos20.1060a/builder:doca-3.4.0", arch: x86_64, build_mode: short }
- { name: "ubuntu2204-rocm6.0", url: "harbor.mellanox.com/ucx/x86_64/ubuntu2204:rocm-6.0.0", arch: x86_64, build_mode: short, CONTAINER: "ubuntu2204_rocm" }
Comment thread
Alexey-Rivkin marked this conversation as resolved.
Resolved
Comment thread
Alexey-Rivkin marked this conversation as resolved.
Resolved
- { name: "centos7-ib", url: "harbor.mellanox.com/ucx/centos7:5", arch: x86_64, build_mode: sanity }
# ZE null driver, no GPU needed.
- { name: "ubuntu2404-ze", url: "harbor.mellanox.com/ucx/x86_64/ubuntu24.04/builder:ze-oneapi-2026.0.0-l0-1.27.0", arch: x86_64, build_mode: ze, require_ze: "yes" }
Comment thread
Alexey-Rivkin marked this conversation as resolved.
Resolved
Comment thread
Alexey-Rivkin marked this conversation as resolved.
Resolved
# ICC/PGI from /hpc/local.
- { name: "rocky89-icc-pgi", url: "harbor.mellanox.com/hpcx/x86_64/rocky8.9/builder:mofed-24.10-3.2.5.0", arch: x86_64, build_mode: compilers }
Comment thread
Alexey-Rivkin marked this conversation as resolved.
Resolved
Comment thread
Alexey-Rivkin marked this conversation as resolved.
Resolved
# aarch64
- { name: "ubuntu2404-doca2.9", url: "harbor.mellanox.com/hpcx/aarch64/ubuntu24.04/builder:doca-2.9.0", arch: aarch64 }
- { name: "ubuntu2604-mofed26.04", url: "harbor.mellanox.com/hpcx/aarch64/ubuntu26.04/builder:mofed-26.04-0.7.6.0", arch: aarch64 }
- { name: "ubuntu2610-inbox", url: "harbor.mellanox.com/hpcx/aarch64/ubuntu26.10/builder:inbox", arch: aarch64 }
- { name: "fedora44-inbox", url: "harbor.mellanox.com/hpcx/aarch64/fedora44/builder:inbox", arch: aarch64 }
- { name: "azurelinux40-inbox", url: "harbor.mellanox.com/hpcx/aarch64/azurelinux4.0/builder:inbox", arch: aarch64 }
- { name: "ubuntu2404-doca2.9", url: "harbor.mellanox.com/hpcx/aarch64/ubuntu24.04/builder:doca-2.9.0", arch: aarch64, build_mode: long }
- { name: "ubuntu2604-mofed26.04", url: "harbor.mellanox.com/hpcx/aarch64/ubuntu26.04/builder:mofed-26.04-0.7.6.0", arch: aarch64, build_mode: short }
- { name: "ubuntu2610-inbox", url: "harbor.mellanox.com/hpcx/aarch64/ubuntu26.10/builder:inbox", arch: aarch64, build_mode: short }
- { name: "fedora44-inbox", url: "harbor.mellanox.com/hpcx/aarch64/fedora44/builder:inbox", arch: aarch64, build_mode: short }
- { name: "azurelinux40-inbox", url: "harbor.mellanox.com/hpcx/aarch64/azurelinux4.0/builder:inbox", arch: aarch64, build_mode: short }
# DOCA publishes no aarch64 openEuler, so these take the MOFED route.
- { name: "openeuler2203sp4-mofed26.07", url: "harbor.mellanox.com/hpcx/aarch64/openeuler22.03sp4/builder:mofed-26.07-0.7.7.0", arch: aarch64 }
- { name: "openeuler2403sp3-mofed26.07", url: "harbor.mellanox.com/hpcx/aarch64/openeuler24.03sp3/builder:mofed-26.07-0.7.7.0", arch: aarch64 }
- { name: "openeuler2203sp4-mofed26.07", url: "harbor.mellanox.com/hpcx/aarch64/openeuler22.03sp4/builder:mofed-26.07-0.7.7.0", arch: aarch64, build_mode: short }
- { name: "openeuler2403sp3-mofed26.07", url: "harbor.mellanox.com/hpcx/aarch64/openeuler24.03sp3/builder:mofed-26.07-0.7.7.0", arch: aarch64, build_mode: short }
# No DOCA or MOFED for UOS 20.1060a on aarch64, so inbox (rdma-core 37).
- { name: "uos201060a-inbox", url: "harbor.mellanox.com/hpcx/aarch64/uos20.1060a/builder:inbox", arch: aarch64 }
- { name: "rocky89-mofed24.10", url: "harbor.mellanox.com/hpcx/aarch64/rocky8.9/builder:mofed-24.10-3.2.5.0", arch: aarch64 }
- { name: "rocky96-mofed24.10", url: "harbor.mellanox.com/hpcx/aarch64/rocky9.6/builder:mofed-24.10-3.2.5.0", arch: aarch64 }
- { name: "rhel100-mofed25.07", url: "harbor.mellanox.com/hpcx/aarch64/rhel10.0/builder:mofed-25.07-0.9.7.0", arch: aarch64 }
- { name: "sles15sp7-mofed26.01", url: "harbor.mellanox.com/hpcx/aarch64/sles15sp7/builder:mofed-26.01-0.8.8.0", arch: aarch64 }
- { name: "ctyunos2507-mofed26.01", url: "harbor.mellanox.com/hpcx/aarch64/ctyunos25.07/builder:mofed-26.01-0.8.0.0", arch: aarch64 }
- { name: "tencentos44-mofed26.04", url: "harbor.mellanox.com/hpcx/aarch64/tencentos4.4/builder:mofed-26.04-0.7.6.0", arch: aarch64 }
- { name: "sles16sp0-mofed26.01", url: "harbor.mellanox.com/hpcx/aarch64/sles16sp0/builder:mofed-26.01-0.8.0.0", arch: aarch64 }
- { name: "uos201060a-inbox", url: "harbor.mellanox.com/hpcx/aarch64/uos20.1060a/builder:inbox", arch: aarch64, build_mode: short }
- { name: "rocky89-mofed24.10", url: "harbor.mellanox.com/hpcx/aarch64/rocky8.9/builder:mofed-24.10-3.2.5.0", arch: aarch64, build_mode: short }
- { name: "rocky96-mofed24.10", url: "harbor.mellanox.com/hpcx/aarch64/rocky9.6/builder:mofed-24.10-3.2.5.0", arch: aarch64, build_mode: short }
- { name: "rhel100-mofed25.07", url: "harbor.mellanox.com/hpcx/aarch64/rhel10.0/builder:mofed-25.07-0.9.7.0", arch: aarch64, build_mode: short }
- { name: "sles15sp7-mofed26.01", url: "harbor.mellanox.com/hpcx/aarch64/sles15sp7/builder:mofed-26.01-0.8.8.0", arch: aarch64, build_mode: short }
- { name: "ctyunos2507-mofed26.01", url: "harbor.mellanox.com/hpcx/aarch64/ctyunos25.07/builder:mofed-26.01-0.8.0.0", arch: aarch64, build_mode: short }
- { name: "tencentos44-mofed26.04", url: "harbor.mellanox.com/hpcx/aarch64/tencentos4.4/builder:mofed-26.04-0.7.6.0", arch: aarch64, build_mode: short }
- { name: "sles16sp0-mofed26.01", url: "harbor.mellanox.com/hpcx/aarch64/sles16sp0/builder:mofed-26.01-0.8.0.0", arch: aarch64, build_mode: short }

taskName: '${arch}/${name}'

env:
BUILD_ID: "${BUILD_NUMBER}"
Comment thread
Alexey-Rivkin marked this conversation as resolved.
Resolved
build_mode: "short"

steps:
- name: Build
parallel: false
timeout: 60
timeout: 180
run: |
# Workspace UID differs from container UID; trust it for git.
git config --global --add safe.directory '*'
Expand Down
170 changes: 0 additions & 170 deletions buildlib/pr/build_job.yml
View file
Open in desktop

This file was deleted.

24 changes: 0 additions & 24 deletions buildlib/pr/main.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -298,30 +298,6 @@ stages:
jobs:
- template: static_checks.yml

- stage: Build
dependsOn: [Basic_compile]
jobs:
- template: build_job.yml
parameters:
name: build_x86_64
demands:
- ucx_docker
displayName: Build on x86_64

- template: build_job.yml
parameters:
name: build_icc_pgi
demands:
- ucx_docker
displayName: Build with ICC and PGI on CentOS7 x86_64

- template: build_job.yml
parameters:
name: build_aarch64
demands:
- ucx_arm64
displayName: Build on aarch64

- stage: ucx_perftest_mad_rte
Comment thread
Alexey-Rivkin marked this conversation as resolved.
Resolved
dependsOn: [Basic_compile]
displayName: ucx_perftest over MAD RTE
Expand Down
Loading