Try to Setup MXnet on Raspberry Pi3 using MXNet installation guide using MAC but fail

[kepler-cpu]

Description

new to programming... trying to setup MXnet on Raspberry pi3 and following the instruction at
https://mxnet.incubator.apache.org/install/index.html?platform=Devices&language=Python&processor=CPU

trying to use MAC to build the build/mxnet-x.x.x-py2.py3-none-any.whl but run into the following problems ; ps: Docker installed on my MAC

Environment info (Required)

----------Python Info----------
('Version :', '2.7.10')
('Compiler :', 'GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.34)')
('Build :', ('default', 'Feb 7 2017 00:08:15'))
('Arch :', ('64bit', ''))
------------Pip Info-----------
No corresponding pip install for current python.
----------MXNet Info-----------
No MXNet installed.
----------System Info----------
('Platform :', 'Darwin-16.7.0-x86_64-i386-64bit')
('system :', 'Darwin')
('node :', 'xxx-MacBook-Pro.local')
('release :', '16.7.0')
('version :', 'Darwin Kernel Version 16.7.0: Thu Jun 21 20:07:39 PDT 2018; root:xnu-3789.73.14~1/RELEASE_X86_64')
----------Hardware Info----------
('machine :', 'x86_64')
('processor :', 'i386')
machdep.cpu.extfeatures: SYSCALL XD 1GBPAGE EM64T LAHF LZCNT RDTSCP TSCI
machdep.cpu.leaf7_features: SMEP ERMS RDWRFSGS TSC_THREAD_OFFSET BMI1 AVX2 BMI2 INVPCID FPU_CSDS
machdep.cpu.features: FPU VME DE PSE TSC MSR PAE MCE CX8 APIC SEP MTRR PGE MCA CMOV PAT PSE36 CLFSH DS ACPI MMX FXSR SSE SSE2 SS HTT TM PBE SSE3 PCLMULQDQ DTES64 MON DSCPL VMX EST TM2 SSSE3 FMA CX16 TPR PDCM SSE4.1 SSE4.2 x2APIC MOVBE POPCNT AES PCID XSAVE OSXSAVE SEGLIM64 TSCTMR AVX1.0 RDRAND F16C
machdep.cpu.brand_string: Intel(R) Core(TM) i5-4258U CPU @ 2.40GHz
----------Network Test----------
Setting timeout: 10
Error open MXNet: https://github.com/apache/incubator-mxnet, <urlopen error [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:590)>, DNS finished in 0.044517993927 sec.
Error open PYPI: https://pypi.python.org/pypi/pip, <urlopen error [SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:590)>, DNS finished in 0.0384840965271 sec.
Timing for FashionMNIST: https://apache-mxnet.s3-accelerate.dualstack.amazonaws.com/gluon/dataset/fashion-mnist/train-labels-idx1-ubyte.gz, DNS: 0.0383 sec, LOAD: 0.3346 sec.
Timing for Conda: https://repo.continuum.io/pkgs/free/, DNS: 0.0496 sec, LOAD: 0.1033 sec.
Error open Gluon Tutorial(en): http://gluon.mxnet.io, <urlopen error [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:590)>, DNS finished in 0.0765810012817 sec.
Error open Gluon Tutorial(cn): https://zh.gluon.ai, <urlopen error [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:590)>, DNS finished in 0.0391330718994 sec.

Package used (Python/R/Scala/Julia):
Python

Build info (Required if built from source)

download the source code from the bottom of the page below:

https://mxnet.incubator.apache.org/install/index.html?platform=Devices&language=Python&processor=CPU

Error Message:

make[2]: /usr/bin/armv7-unknown-linux-gnueabi-gfortran: Command not found
/usr/bin/armv7-unknown-linux-gnueabi-gfortran -O2 -marm -Wall -fPIC -mfpu=vfpv3 -march=armv7-a -c -o lsametst.o lsametst.f
make[2]: /usr/bin/armv7-unknown-linux-gnueabi-gfortran: Command not found
Makefile:32: recipe for target 'lsame.o' failed
make[2]: *** [lsame.o] Error 127
make[2]: *** Waiting for unfinished jobs....
Makefile:32: recipe for target 'lsametst.o' failed
make[2]: Leaving directory '/work/deps/OpenBLAS/lapack-netlib/INSTALL'
make[2]: *** [lsametst.o] Error 127
make[1]: *** [lapack_install] Error 2
make: *** [netlib] Error 2
Makefile:17: recipe for target 'lapack_install' failed
make[1]: Leaving directory '/work/deps/OpenBLAS/lapack-netlib'
Makefile:215: recipe for target 'netlib' failed
The command '/bin/sh -c /work/arm_openblas.sh' returned a non-zero code: 2
Traceback (most recent call last):
File "ci/build.py", line 252, in
sys.exit(main())
File "ci/build.py", line 186, in main
build_docker(platform, docker_binary)
File "ci/build.py", line 73, in build_docker
check_call(cmd)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/subprocess.py", line 291, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['docker', 'build', '-f', 'docker/Dockerfile.build.armv7', '--build-arg', 'USER_ID=501', '-t', 'mxnet/build.armv7', 'docker']' returned non-zero exit status 2.

Minimum reproducible example

follow the instruction at
https://mxnet.incubator.apache.org/install/index.html?platform=Devices&language=Python&processor=CPU

when executing ci/build.py -p armv7

Steps to reproduce

ci/build.py -p armv7 from MAC terminal after Docker is installed

What have you tried to solve it?

search for Google but didn't find how exactly yet... some indicates missing fortran complier but don't know how to address that and the discussion indicates something more may need to be done.. maybe need to change Dockfile?
also, it seems some cross compilation issue... try to install dock cross but didn't work

[vrakesh]

@kepler-cpu Thank you for reporting the issue, I tried out the docker build path. I am also running into the issue. I am trying various versions of the source repo to see if any of them builds with the docker path.

Also requesting you to try a native build on rasberry pi to see if that works.

@mxnet-label=bot [Bug, Build, Breaking]

[vrakesh]

@mxnet-label-bot [Bug, Build, Breaking]

[kepler-cpu]

Thanks @vrakesh... I am trying the native build as suggested... currently running into the following error msg:
//
liblapack3 set to manually installed.
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
liblapack3 : Breaks: liblapack3gf
E: Unable to correct problems, you have held broken packages.
//

will try to troubleshoot it but if you have any insights, please let me know too. thx!

[kepler-cpu]

try native build on pi directly but still get errors when executing the cmake \ commands in the 'Build' section from the instruction

below is the errors I got...
ps: have tried to install the ninja-built but still fail
ninja --version shows 1.3.4
cmake version shows 3.6.2

appreciate any guidance on how to move this forward...

Error Msg:

pi@test:~/incubator-mxnet/build $ cmake \

    -DUSE_SSE=OFF \
    -DUSE_CUDA=OFF \
    -DUSE_OPENCV=ON \
    -DUSE_OPENMP=ON \
    -DUSE_MKL_IF_AVAILABLE=OFF \
    -DUSE_SIGNAL_HANDLER=ON \
    -DCMAKE_BUILD_TYPE=Release \
    -GNinja ..

-- The C compiler identification is GNU 4.9.2
-- The CXX compiler identification is GNU 4.9.2
-- Check for working C compiler: /usr/bin/cc
CMake Error:
The detected version of Ninja () is less than the version of Ninja required
by CMake (1.3).

CMake Error: Internal CMake error, TryCompile generation of cmake failed
-- Check for working C compiler: /usr/bin/cc -- broken
CMake Error at /usr/share/cmake-3.6/Modules/CMakeTestCCompiler.cmake:61 (message):
The C compiler "/usr/bin/cc" is not able to compile a simple test program.

It fails with the following output:

CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
CMakeLists.txt:3 (project)

-- Configuring incomplete, errors occurred!
See also "/home/pi/incubator-mxnet/build/CMakeFiles/CMakeOutput.log".
See also "/home/pi/incubator-mxnet/build/CMakeFiles/CMakeError.log".

[marcoabreu]

Hello, have you tried running our Docker based build pipeline? You can execute it using 'ci/build.py -p armv7'

[kepler-cpu]

if I get your point, I think my answer is yes... I tried the ci/build.py -p armv7 script that I download to my MAC from the source at https://mxnet.incubator.apache.org/install/index.html?platform=Devices&language=Python&processor=CPU

Q. I tried that build on my MAC. I assume it's ok. or I should use another type of platform?

However, I got a list of errors. Please see my 1st post.
It seems the 1st error is not finding gfrortran command?

make[2]: /usr/bin/armv7-unknown-linux-gnueabi-gfortran: Command not found

Should I modify the build.py script?

Thanks!

[kepler-cpu]

want to update this case and share some my experiences with the community...

I have finally successfully cross-complied the mxnet for armv7 (Pi) - the whl file instructed in the mxnet doc. The way I resolved it is using the latest 1.3.0 source instead of the 1.2.1 that gave me problems. thx the the CI build from the mxnet team I guess :-)

I am following the next step to install the whl package with pip command under a virtualenv.
It looks the mxnet package is installed successfully too.

However, when I tried to test by import mxnet as mx under python interpreter, I got an error indicating:
OSError: /... path to libmxnet.so : cannot open shared object file. No sun file or directory

To get around this, I add the path to the libmxnet.so using:

import sys
sys.path.append(path to the libmxnet.so)

alternatively, I would need to run the python interpreter in the same directory of libmxnet.so.

(btw, I saw the sys.path has this path - '/home/pi/my_mxnet_work/mxnet_py3/lib/python3.4/site-packages, inserted in the end. I assume that's added by the build script. if so, I am wondering if the script should add '/home/pi/my_mxnet_work/mxnet_py3/lib/python3.4/site-packages/mxnet' in the path as well? )

However, I run into another problem about ImportError - see output below.
I saw quiet few discussions about this error in the past but still not sure how to resolve it yet.

I will dig out more but just to update this case in case someone has insights

//

import mxnet as mx
Traceback (most recent call last):
File "", line 1, in
File "/home/pi/my_mxnet_work/mxnet_py3/lib/python3.4/site-packages/mxnet/init.py", line 24, in
from .context import Context, current_context, cpu, gpu, cpu_pinned
File "/home/pi/my_mxnet_work/mxnet_py3/lib/python3.4/site-packages/mxnet/context.py", line 24, in
from .base import classproperty, with_metaclass, _MXClassPropertyMetaClass
File "/home/pi/my_mxnet_work/mxnet_py3/lib/python3.4/site-packages/mxnet/base.py", line 214, in
_LIB = _load_lib()
File "/home/pi/my_mxnet_work/mxnet_py3/lib/python3.4/site-packages/mxnet/base.py", line 205, in _load_lib
lib = ctypes.CDLL(lib_path[0], ctypes.RTLD_LOCAL)
File "/usr/lib/python3.4/ctypes/init.py", line 351, in init
self._handle = _dlopen(self._name, mode)
OSError: /home/pi/my_mxnet_work/mxnet_py3/lib/python3.4/site-packages/mxnet/libmxnet.so: cannot open shared object file: No such file or directory

import sys
print (sys.path)
['', '/home/pi/my_mxnet_work/mxnet_py3/lib/python3.4', '/home/pi/my_mxnet_work/mxnet_py3/lib/python3.4/plat-arm-linux-gnueabihf', '/home/pi/my_mxnet_work/mxnet_py3/lib/python3.4/lib-dynload', '/usr/lib/python3.4', '/usr/lib/python3.4/plat-arm-linux-gnueabihf', '/home/pi/my_mxnet_work/mxnet_py3/lib/python3.4/site-packages']
sys.path.append('/home/pi/my_mxnet_work/mxnet_py3/lib/python3.4/site-packages/mxnet')
print (sys.path)
['', '/home/pi/my_mxnet_work/mxnet_py3/lib/python3.4', '/home/pi/my_mxnet_work/mxnet_py3/lib/python3.4/plat-arm-linux-gnueabihf', '/home/pi/my_mxnet_work/mxnet_py3/lib/python3.4/lib-dynload', '/usr/lib/python3.4', '/usr/lib/python3.4/plat-arm-linux-gnueabihf', '/home/pi/my_mxnet_work/mxnet_py3/lib/python3.4/site-packages', '/home/pi/my_mxnet_work/mxnet_py3/lib/python3.4/site-packages/mxnet']
import mxnet as mx
Traceback (most recent call last):
File "", line 1, in
File "/home/pi/my_mxnet_work/mxnet_py3/lib/python3.4/site-packages/mxnet/init.py", line 24, in
from .context import Context, current_context, cpu, gpu, cpu_pinned
File "/home/pi/my_mxnet_work/mxnet_py3/lib/python3.4/site-packages/mxnet/context.py", line 24, in
from .base import classproperty, with_metaclass, _MXClassPropertyMetaClass
File "/home/pi/my_mxnet_work/mxnet_py3/lib/python3.4/site-packages/mxnet/base.py", line 31, in
from . import libinfo
ImportError: cannot import name 'libinfo'

[kepler-cpu]

what is the version of python that the mxnet package is supported on Pi?

I am wondering if there is a version or modules path issues in my setting.
It depends on which directory I enter python, I got different types of Import Error.

The early reported one - by entering python under /home/pi/my_mxnet_work/mxnet_py3, and append a path (see above), I got the ImportError about lib info.

Now, by entering python under ... /home/pi/my_mxnet_work/mxnet_py3/lib/python3.4/site_-packages/mxnet, I got:

File "/home/pi/my_mxnet_work/mxnet_py3/lib/python3.4/site-packages/mxnet/operator.py", line 24 ,in
ImportError: cannot import name 'Lock'

If I pick another directory, I got another import error.

I think maybe I miss something fundamental?

Any input?

thx!

[lebeg]

@kepler-cpu the runtime on RPi was broken, but should be fixed when #12588 will be merged. Regarding your initial problem it's related to fortran not being present in dockcross and was fixed with dockcross/dockcross#228. The fix should be contained in our custom builds on dockerhub https://hub.docker.com/r/mxnetci/.

[kepler-cpu]

Thanks @lebeg! (as a beginner, I feel a slight better now :)

so to get the fixed, should I just wait for the newer version of source to appear from the following?
https://mxnet.incubator.apache.org/install/download.html

My first fortran error was not appeared when I use 1.3.0 version. So I assume the docker cross issue#228 is fixed already in the 1.3.0? If so, I should wait for #12588 fix in the next newer source, and I can follow the same instruction from the Installing MXNET doc., right?

thx!

btw, just in case it matters... when I track Lock import error back to the threading.py file. I cannot see Lock function but RLock function. Even if I modified the operator.py to use RLock, I still get Import Error. just FYI

[vrakesh]

@mxnet-label-bot [ARM]

[lebeg]

@kepler-cpu I'm not sure about the lock problem, didn't see anything like that before. Maybe you could try using master build with

ci/build.py -p armv7

The artifact should be available afterwards in ./build. Try to pip install it and see if it works.

[piyushghai]

@kepler-cpu Did the command ci/build.py -p armv7 help ?

[kepler-cpu]

sorry for the late reply... to answer your question, I did native build in the past with the command you suggested but it didn't work in the end. It did built the lib libmxnet.so but when I tried to use it under python, I got import error. (pls see my past post). ps: to move my project forward, I am adopting TF and it seems working so far on Pi.