Skip to content
This repository has been archived by the owner on Nov 17, 2023. It is now read-only.

MXNet with MKLDNN failed to install horovod #15211

Closed
roywei opened this issue Jun 11, 2019 · 2 comments · Fixed by #15213
Closed

MXNet with MKLDNN failed to install horovod #15211

roywei opened this issue Jun 11, 2019 · 2 comments · Fixed by #15213
Labels
Breaking Distributed Installation MKLDNN

Comments

@roywei
Copy link
Member

roywei commented Jun 11, 2019 •
edited
Loading

From https://lists.apache.org/thread.html/6b498b466123b11df023f746fb233117bc58a68818383b0952a9c9e2@%3Cdev.mxnet.apache.org%3E

I was able to verify and reproduce the error.

using MXNet built from source is not able to install Horovod

Steps to reproduce:
Using AWS Base DLAMI on P3.2xLarge.
checkout tag 1.5.0.rc0

sudo make -j$(nproc) USE_CUDA=1 USE_CUDA_PATH=/usr/local/cuda USE_CUDNN=1 USE_NCCL=1 USE_BLAS=openblas

cd python
sudo pip install -e .

Able to build successfully and run a simple mnist example.

HOROVOD_GPU_ALLREDUCE=NCCL  pip install horovod --user --no-cache-dir -U

error

home/ubuntu/src/incubator-mxnet/python/mxnet/../../include/mkldnn/mkldnn.h:55:28:
fatal error: mkldnn_version.h: No such file or directory
    compilation terminated.
    INFO: Unable to build MXNet plugin, will skip it.
@mxnet-label-bot
Copy link
Contributor

Hey, this is the MXNet Label Bot.
Thank you for submitting the issue! I will try and suggest some labels so that the appropriate MXNet community members can help resolve it.
Here are my recommended labels: Installation

@roywei roywei changed the title MXNet built from source does not work with horovod MXNet with MKLDNN does not work with horovod Jun 11, 2019
@roywei
Copy link
Member Author

roywei commented Jun 11, 2019 •
edited
Loading

@mxnet-label-bot add[MKLDNN, Distributed, Installation, Breaking]

@roywei roywei changed the title MXNet with MKLDNN does not work with horovod MXNet with MKLDNN failed to install horovod Jun 11, 2019
@apeforest apeforest mentioned this issue Jun 11, 2019
Merged
@roywei roywei mentioned this issue Jun 16, 2019
Open
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
Breaking Distributed Installation MKLDNN
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix horovod build failure when mxnet is built from source apeforest/incubator-mxnet
3 participants
@roywei @marcoabreu @mxnet-label-bot