Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

[v1.x] use centos7 base image for aarch64 build #20392

Merged
merged 14 commits into from
Jul 6, 2021

Conversation

mseth10
Copy link
Contributor

@mseth10 mseth10 commented Jun 28, 2021

Description

This PR changes the base docker image used by CD pipeline from Ubuntu 18 to CentOS 7. This relaxes the system glibc requirement of the nightly wheel from >=2.27 to >=2.17, thus allowing it to run on all manylinux2014 systems.

We install devtoolset-10 and rh-python38 packages from CentOS SCLo RH repository, which gives gcc/g++/gfortran-10.2 allowing us to use -moutline-atomics C/C++ build flag, and numpy-v1.17.3 which helps pass unittests which were failing earlier #20289.

We continue to use ArmPL built with gcc-8 as ArmPL built with gcc-10 requires some gfortran-10 symbols that are not present in devtoolset-10 as it is a patched version.

The changes have been tested on a test CD pipeline and it passes for all flavors: https://jenkins.mxnet-ci.amazon-ml.com/blue/organizations/jenkins/restricted-mxnet-cd%2Fsethman-test-cd-pipeline/detail/sethman-test-cd-pipeline/18/pipeline/30

@mxnet-bot
Copy link

Hey @mseth10 , Thanks for submitting the PR
All tests are already queued to run once. If tests fail, you can trigger one or more tests again with the following commands:

  • To trigger all jobs: @mxnet-bot run ci [all]
  • To trigger specific jobs: @mxnet-bot run ci [job1, job2]

CI supported jobs: [windows-cpu, clang, edge, miscellaneous, unix-cpu, centos-cpu, unix-gpu, windows-gpu, centos-gpu, website, sanity]


Note:
Only following 3 categories can trigger CI :PR Author, MXNet Committer, Jenkins Admin.
All CI tests must pass before the PR can be merged.

@mseth10 mseth10 added the pr-work-in-progress PR is still work in progress label Jun 28, 2021
@mseth10 mseth10 force-pushed the centos7-aarch64 branch 8 times, most recently from ccc57fa to c28212e Compare July 3, 2021 11:11
@mseth10 mseth10 changed the title [WIP] use centos7 base image for aarch64 build [v1.x] use centos7 base image for aarch64 build Jul 5, 2021
@mseth10 mseth10 requested review from Zha0q1, leezu and szha July 5, 2021 21:55
@mseth10 mseth10 added pr-awaiting-testing PR is reviewed and waiting CI build and test and removed pr-work-in-progress PR is still work in progress labels Jul 5, 2021
@mseth10 mseth10 added pr-work-in-progress PR is still work in progress and removed pr-awaiting-testing PR is reviewed and waiting CI build and test labels Jul 6, 2021
@mseth10
Copy link
Contributor Author

mseth10 commented Jul 6, 2021

@mxnet-bot run ci [centos-cpu, centos-gpu, clang, edge, unix-cpu, unix-gpu, miscellaneous]

@mxnet-bot
Copy link

Jenkins CI successfully triggered : [centos-gpu, unix-cpu, clang, centos-cpu, edge, miscellaneous, unix-gpu]

@mseth10 mseth10 added pr-awaiting-testing PR is reviewed and waiting CI build and test and removed pr-work-in-progress PR is still work in progress labels Jul 6, 2021
Copy link
Contributor

@Zha0q1 Zha0q1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks Manu!

@mseth10 mseth10 added pr-awaiting-merge Review and CI is complete. Ready to Merge and removed pr-awaiting-testing PR is reviewed and waiting CI build and test labels Jul 6, 2021
Copy link
Contributor

@waytrue17 waytrue17 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks!

@mseth10 mseth10 merged commit 962a8ae into apache:v1.x Jul 6, 2021
@mseth10 mseth10 added ARM CD and removed pr-awaiting-merge Review and CI is complete. Ready to Merge labels Jul 6, 2021
waytrue17 pushed a commit to waytrue17/incubator-mxnet that referenced this pull request Jul 7, 2021
* add centos7 aarch64 build

* enable rh packages

* update openssl curl version

* install rhel7 armpl

* find armpl cmake

* test mkldnn build

* test ninja

* use armpl compiled with gcc-8

* enable py38 for unittests

* check previously failing tests

* create symlink for ninja

* fix cd pypi

* fix cd docker

* update ssl for mac
mseth10 added a commit that referenced this pull request Jul 7, 2021
…0423)

* [v1.x] use centos7 base image for aarch64 build (#20392)

* add centos7 aarch64 build

* enable rh packages

* update openssl curl version

* install rhel7 armpl

* find armpl cmake

* test mkldnn build

* test ninja

* use armpl compiled with gcc-8

* enable py38 for unittests

* check previously failing tests

* create symlink for ninja

* fix cd pypi

* fix cd docker

* update ssl for mac

* [v1.x] Update CD pipeline to build with centos7 (#20409)

Co-authored-by: Wei Chu <[email protected]>

Co-authored-by: Manu Seth <[email protected]>
Co-authored-by: Wei Chu <[email protected]>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants