-
Notifications
You must be signed in to change notification settings - Fork 6.8k
Conversation
prepare_mkldnn.sh
Outdated
make -C $MKLDNN_BUILDDIR -j$(sysctl -n hw.ncpu) VERBOSE=1 >&2 | ||
else | ||
make -C $MKLDNN_BUILDDIR VERBOSE=1 >&2 | ||
fi |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe refactor the processor discovery to reduce duplicated make lines? I used this snippet for discovering number of processes elsewhere.
NUM_PROC=1
if [[ ! -z $(command -v nproc) ]]; then
NUM_PROC=$(nproc)
elif [[ ! -z $(command -v sysctl) ]]; then
NUM_PROC=$(sysctl -n hw.ncpu)
else
>&2 echo "Can't discover number of cores."
fi
prepare_mkldnn.sh
Outdated
@@ -84,7 +84,13 @@ if [ ! -f "$MKLDNN_INSTALLDIR/lib/libmkldnn.so" ]; then | |||
cd $MXNET_ROOTDIR | |||
g++ --version >&2 | |||
cmake $MKLDNN_ROOTDIR -DCMAKE_INSTALL_PREFIX=$MKLDNN_INSTALLDIR -B$MKLDNN_BUILDDIR -DARCH_OPT_FLAGS="-mtune=generic" >&2 | |||
make -C $MKLDNN_BUILDDIR -j$(cat /proc/cpuinfo | grep processor | wc -l) VERBOSE=1 >&2 | |||
if [ $OSTYPE == "linux-gnu" ]; then | |||
make -C $MKLDNN_BUILDDIR -j$(cat /proc/cpuinfo | grep processor | wc -l) VERBOSE=1 >&2 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think -j$(nproc) will work on most platforms
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
make -j$(nproc)
doesn't seem to work on Mac
$ make -j$(nproc)
-bash: nproc: command not found```
prepare_mkldnn.sh
Outdated
elif [ $OSTYPE == "darwin16" ]; then | ||
make -C $MKLDNN_BUILDDIR -j$(sysctl -n hw.ncpu) VERBOSE=1 >&2 | ||
else | ||
make -C $MKLDNN_BUILDDIR VERBOSE=1 >&2 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Which OS is the else branch? Why multiple cores can not be used?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I only listed two popular OS. I don't know what other OS people use and what mechanism is provided for getting the number of cores automatically. We need to handle the default case.
Given that our CI doesn't have mac yet, please verify the new logic locally. |
I tested it. It works fine. |
Does the current logic require re-building mkldnn at every |
Yes, it's weird. We don't need to rebuild mkldnn every time in Linux. |
This needs to be fixed. |
it should work now. |
FYI, as @KellenSunderland @marcoabreu 's suggestion, we added Clang + MKLDNN instance in CI process #9918 (#9828) to track the OSX building issues. |
clang build is different from building on mac. For example, clang build won't catch the processor detection problem. |
* Compile MKLDNN in Mac. * A more general way of finding #cores. * find the right dyn library of mkldnn.
* Compile MKLDNN in Mac. * A more general way of finding #cores. * find the right dyn library of mkldnn.
* Compile MKLDNN in Mac. * A more general way of finding #cores. * find the right dyn library of mkldnn.
Description
This is to make MKLDNN compiled in Mac correctly.
Checklist
Essentials
make lint
)