Install on Mavericks (Mac OS X 10.9) #209

lazywei · 2014-03-16T02:31:44Z

I tried to install latest NMatrix on my Macbook (OS X 10.9), and I followed the installation instruction on wiki

I have gcc 4.9 installed

$ gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/local/Cellar/gcc49/4.9-20140302/libexec/gcc/x86_64-apple-darwin13.1.0/4.9.0/lto-wrapper
Target: x86_64-apple-darwin13.1.0
Configured with: ../configure --build=x86_64-apple-darwin13.1.0 --prefix=/usr/local/Cellar/gcc49/4.9-20140302 --enable-languages=c,c++,objc,obj-c++,fortran --program-suffix=-4.9 --with-gmp=/usr/local/opt/gmp4 --with-mpfr=/usr/local/opt/mpfr2 --with-mpc=/usr/local/opt/libmpc08 --with-cloog=/usr/local/opt/cloog018 --with-isl=/usr/local/opt/isl011 --with-system-zlib --enable-version-specific-runtime-libs --enable-libstdcxx-time=yes --enable-stage1-checking --enable-checking=release --enable-lto --disable-werror --enable-plugin --disable-nls --enable-multilib
Thread model: posix
gcc version 4.9.0 20140302 (experimental) (GCC)

I have linked clapack.h and cblas.h to /usr/include/

$ ls -l /usr/include/{clapack.h,cblas.h}
lrwxr-xr-x  1 root  wheel  125 Mar 16 00:13 /usr/include/cblas.h -> /System/Library/Frameworks/Accelerate.framework/Versions/Current/Frameworks/vecLib.framework/Versions/Current/Headers/cblas.h
lrwxr-xr-x  1 root  wheel  127 Mar 16 00:19 /usr/include/clapack.h -> /System/Library/Frameworks/Accelerate.framework/Versions/Current/Frameworks/vecLib.framework/Versions/Current/Headers/clapack.h

However, when I tried to compile, I got this

$ rake clean
$ rake compile
mkdir -p tmp/x86_64-darwin13.0/nmatrix/2.1.0
cd tmp/x86_64-darwin13.0/nmatrix/2.1.0
/Users/lazywei/.rbenv/versions/2.1.0/bin/ruby -I. ../../../../ext/nmatrix/extconf.rb
using C++ standard... c++11
g++ reports version... 4.9.0
checking for main() in -lclapack... yes
checking for main() in -llapack... yes
checking for main() in -lcblas... yes
checking for main() in -latlas... no
checking for atlas/cblas.h... no
checking for cblas.h... yes
checking for clapack.h... yes
checking for clapack_dgetrf() in cblas.h,clapack.h... no
checking for clapack_dgetri() in cblas.h,clapack.h... no
checking for dgesvd_() in clapack.h... yes
checking for cblas_dgemm() in cblas.h... yes
creating nmatrix_config.h
creating Makefile
cd -
cd tmp/x86_64-darwin13.0/nmatrix/2.1.0
make
compiling ../../../../ext/nmatrix/nmatrix.cpp
In file included from ../../../../ext/nmatrix/nmatrix.cpp:59:0:
../../../../ext/nmatrix/math/math.h: In function 'int nm::math::potrf(CBLAS_ORDER, CBLAS_UPLO, int, DType*, int) [with DType = float]':
../../../../ext/nmatrix/math/math.h:552:47: error: 'clapack_spotrf' was not declared in this scope
   return clapack_spotrf(order, uplo, N, A, lda);

It said that I have no clapack_spotrf in my header file. I checked my clapack.h, there is a int spotrf_(...) but no clapack_spotrf.

I'm not sure if there is anyone else who compile NMatrix on mavericks successfully?
I did a little search, I'm not sure if #188 is related to this one. I've also commented in here.

The text was updated successfully, but these errors were encountered:

translunar · 2014-03-18T00:13:04Z

#188 is unrelated. You should have multiple clapack.h files. Have you checked all, or just the one in /usr/local/atlas/include/?

lazywei · 2014-03-18T03:04:04Z

There is no /usr/local/atlas folder in my laptop. I only have clapack.h in

/System/Library/Frameworks/Accelerate.framework/Versions/Current/Frameworks/\
vecLib.framework/Versions/Current/Headers/

translunar · 2014-03-18T20:36:35Z

That's bizarre. Are you sure you've properly installed developer tools? I'm not quite sure why this would happen.

thisMagpie · 2014-04-05T19:51:38Z

It seems there has also been a similar problem for lion users

maybe also see?

translunar · 2014-04-05T20:54:47Z

Try creating the directory and linking the necessary libraries.

On Saturday, April 5, 2014, thisMagpie [email protected] wrote:

It seems there has also been a similar problem for lion usershttps://icl.cs.utk.edu/lapack-forum/viewtopic.php?f=4&t=3109#p10305

maybe also see? http://www.cplusplus.com/forum/unices/43540/

Reply to this email directly or view it on GitHubhttps://github.com//issues/209#issuecomment-39649001
.

andrewcsmith · 2014-04-08T15:41:01Z

+1 for this, and say that I'm having a very similar problem. The difference is that mine is complaining about a lack of clapack_spotrf, clapack_dpotrf, and others.

The error message is pasted below, but I just wanted to add that I once successfully NMatrix this using a manually installed version of Atlas, and since moved from macports to homebrew where I've tried to use the Mavericks scripts provided. My symlinks are all set up as well:

$ ls -l | grep cblas.h
lrwxr-xr-x    1 root  wheel     125 Mar 28  2013 cblas.h -> /System/Library/Frameworks/Accelerate.framework/Versions/Current/Frameworks/vecLib.framework/Versions/Current/Headers/cblas.h
$ ls -l | grep clapack.h
lrwxr-xr-x    1 root  wheel     127 Oct 29 22:08 clapack.h -> /System/Library/Frameworks/Accelerate.framework/Versions/Current/Frameworks/vecLib.framework/Versions/Current/Headers/clapack.h

Finally, here's the full error message:

[edit: This is on gcc-4.7.3, but I get an identical error message to @lazywei when using 4.9.]

In file included from ../../../../ext/nmatrix/nmatrix.cpp:59:0:
../../../../ext/nmatrix/math/math.h: In function 'int nm::math::potrf(CBLAS_ORDER, CBLAS_UPLO, int, DType*, int) [with DType = float]':
../../../../ext/nmatrix/math/math.h:552:47: error: 'clapack_spotrf' was not declared in this scope
../../../../ext/nmatrix/math/math.h: In function 'int nm::math::potrf(CBLAS_ORDER, CBLAS_UPLO, int, DType*, int) [with DType = double]':
../../../../ext/nmatrix/math/math.h:557:47: error: 'clapack_dpotrf' was not declared in this scope
../../../../ext/nmatrix/math/math.h: In function 'int nm::math::potrf(CBLAS_ORDER, CBLAS_UPLO, int, DType*, int) [with DType = nm::Complex<float>]':
../../../../ext/nmatrix/math/math.h:562:72: error: 'clapack_cpotrf' was not declared in this scope
../../../../ext/nmatrix/math/math.h: In function 'int nm::math::potrf(CBLAS_ORDER, CBLAS_UPLO, int, DType*, int) [with DType = nm::Complex<double>]':
../../../../ext/nmatrix/math/math.h:567:72: error: 'clapack_zpotrf' was not declared in this scope
../../../../ext/nmatrix/math/math.h: In function 'void nm::math::lauum(CBLAS_ORDER, CBLAS_UPLO, int, float*, int)':
../../../../ext/nmatrix/math/math.h:941:40: error: there are no arguments to 'clapack_slauum' that depend on a template parameter, so a declaration of 'clapack_slauum' must be available [-fpermissive]
../../../../ext/nmatrix/math/math.h:941:40: note: (if you use '-fpermissive', G++ will accept your code, but allowing the use of an undeclared name is deprecated)
../../../../ext/nmatrix/math/math.h: In function 'void nm::math::lauum(CBLAS_ORDER, CBLAS_UPLO, int, double*, int)':
../../../../ext/nmatrix/math/math.h:946:40: error: there are no arguments to 'clapack_dlauum' that depend on a template parameter, so a declaration of 'clapack_dlauum' must be available [-fpermissive]
../../../../ext/nmatrix/math/math.h: In function 'void nm::math::lauum(CBLAS_ORDER, CBLAS_UPLO, int, nm::Complex64*, int)':
../../../../ext/nmatrix/math/math.h:951:40: error: there are no arguments to 'clapack_clauum' that depend on a template parameter, so a declaration of 'clapack_clauum' must be available [-fpermissive]
../../../../ext/nmatrix/math/math.h: In function 'void nm::math::lauum(CBLAS_ORDER, CBLAS_UPLO, int, nm::Complex128*, int)':
../../../../ext/nmatrix/math/math.h:956:40: error: there are no arguments to 'clapack_zlauum' that depend on a template parameter, so a declaration of 'clapack_zlauum' must be available [-fpermissive]
../../../../ext/nmatrix/math/math.h: In function 'int nm::math::potri(CBLAS_ORDER, CBLAS_UPLO, int, DType*, int) [with DType = float]':
../../../../ext/nmatrix/math/math.h:1032:47: error: 'clapack_spotri' was not declared in this scope
../../../../ext/nmatrix/math/math.h: In function 'int nm::math::potri(CBLAS_ORDER, CBLAS_UPLO, int, DType*, int) [with DType = double]':
../../../../ext/nmatrix/math/math.h:1037:47: error: 'clapack_dpotri' was not declared in this scope
../../../../ext/nmatrix/math/math.h: In function 'int nm::math::potri(CBLAS_ORDER, CBLAS_UPLO, int, DType*, int) [with DType = nm::Complex<float>]':
../../../../ext/nmatrix/math/math.h:1042:72: error: 'clapack_cpotri' was not declared in this scope
../../../../ext/nmatrix/math/math.h: In function 'int nm::math::potri(CBLAS_ORDER, CBLAS_UPLO, int, DType*, int) [with DType = nm::Complex<double>]':
../../../../ext/nmatrix/math/math.h:1047:72: error: 'clapack_zpotri' was not declared in this scope

translunar · 2014-04-08T15:45:49Z

@andrewcsmith Is this in /usr/local/atlas? You didn't say.

andrewcsmith · 2014-04-08T15:49:53Z

@MohawkJohn No, it's not. This is /usr/include/cblas.h and /usr/include/clapack.h linked to the Accelerate headers.

translunar · 2014-04-08T16:17:58Z

Okay, do you have your libs properly linked as well? For example, I have these:

Johns-MacBook-Air-2:lib jwoods$ ls -l /usr/lib/libcblas.dylib
lrwxr-xr-x  1 root  wheel   116B Nov 13 18:04 /usr/lib/libcblas.dylib@ -> ../../System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
Johns-MacBook-Air-2:lib jwoods$ ls -l /usr/lib/libatlas.dylib
lrwxr-xr-x  1 root  wheel   121B Nov 15 14:10 /usr/lib/libatlas.dylib@ -> /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk/usr/lib/libatlas.dylib
Johns-MacBook-Air-2:lib jwoods$ ls -l /usr/lib/libclapack.dylib
lrwxr-xr-x  1 root  wheel   118B Nov 13 18:04 /usr/lib/libclapack.dylib@ -> ../../System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libLAPACK.dylib
Johns-MacBook-Air-2:lib jwoods$ ls -l /usr/lib/liblapack.dylib
lrwxr-xr-x  1 root  wheel   118B Nov 13 18:04 /usr/lib/liblapack.dylib@ -> ../../System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libLAPACK.dylib

andrewcsmith · 2014-04-08T16:36:05Z

Yes, I have all those. My only difference is that mine don't have the @ ->, just the ->.

$ ls -l /usr/lib/libcblas.dylib
lrwxr-xr-x  1 root  wheel  116 Feb  9 21:39 /usr/lib/libcblas.dylib -> ../../System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
$ ls -l /usr/lib/libatlas.dylib
lrwxr-xr-x  1 root  wheel  121 Apr  7 21:16 /usr/lib/libatlas.dylib -> /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.8.sdk/usr/lib/libatlas.dylib
$ ls -l /usr/lib/libclapack.dylib
lrwxr-xr-x  1 root  wheel  118 Feb  9 21:39 /usr/lib/libclapack.dylib -> ../../System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libLAPACK.dylib
$ ls -l /usr/lib/liblapack.dylib
lrwxr-xr-x  1 root  wheel  118 Feb  9 21:39 /usr/lib/liblapack.dylib -> ../../System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libLAPACK.dylib

translunar · 2014-04-08T17:02:39Z

Can you do a locate libatlas.a for me?

andrewcsmith · 2014-04-08T17:08:58Z

What a glorious mess of macports, brew, and manual installation this is:

/Users/acsmith/lib/atlas/lib/libatlas.a
/Users/acsmith/workspaces/ATLAS/build/lib/libatlas.a
/Users/acsmith/workspaces/ATLAS3.10.1/OSX/lib/libatlas.a
/opt/local/lib/libatlas.a
/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_math_atlas/atlas/work/atlas-3.10.1/build/lib/libatlas.a
/usr/local/atlas-3.10.1/lib/libatlas.a
/usr/local/atlas-old/OSX/lib/libatlas.a
/usr/local/atlas-old/build/lib/libatlas.a

translunar · 2014-04-08T17:14:50Z

Yes. That's probably the source of your problem. Here's my list:

$ locate libatlas.a
/usr/local/atlas/lib/libatlas.a

andrewcsmith · 2014-04-08T17:23:38Z

Does the order matter? I'm not sure why my libatlas.a file isn't showing up, because I definitely have one in /usr/local/atlas/lib/libatlas.a as well.

I tried adding it to the beginning of the PATH variable, but that didn't help either.

translunar · 2014-04-08T17:36:42Z

I don't know. This line kind of worries me: checking for main() in -latlas... no

It's not finding the correct atlas. You could try using the nm command to check each of those libatlas files for main and see which ones match and which don't. That'd help you narrow down which one it might be looking at first.

translunar · 2014-04-08T17:38:50Z

FYI, here's what mine looks like:

 bundle exec rake compile
mkdir -p tmp/x86_64-darwin13.0.0/nmatrix/2.0.0
cd tmp/x86_64-darwin13.0.0/nmatrix/2.0.0
/usr/local/var/rbenv/versions/2.0.0-p247/bin/ruby -I. ../../../../ext/nmatrix/extconf.rb
checking for apparent GNU g++ binary with C++0x/C++11 support... 4.9
using C++ standard... c++11
g++ reports version... 4.9.0
checking for main() in -lclapack... yes
checking for main() in -llapack... yes
checking for main() in -lcblas... yes
checking for main() in -latlas... yes
checking for atlas/cblas.h... no
checking for cblas.h... yes
checking for clapack.h... no
checking for clapack_dgetrf() in cblas.h,clapack.h... no
checking for clapack_dgetri() in cblas.h,clapack.h... no
checking for dgesvd_() in clapack.h... no
checking for cblas_dgemm() in cblas.h... yes
creating nmatrix_config.h
creating Makefile
cd -
cd tmp/x86_64-darwin13.0.0/nmatrix/2.0.0
make

andrewcsmith · 2014-04-08T18:56:31Z

Okay — actually, I gave it another shot and I do have main() in -latlas after all.

I think the problem is stemming from the line in the installation wiki, "We want to include cblas.h but not clapack.h." This is because all the errors I'm getting are for symbols clapack_(function name). Also, because I have this, which indicates that my search path is finding dgesvd() in clapack.h (and yours does not).

g++ reports version... 4.9.0
checking for main() in -lclapack... yes
checking for main() in -llapack... yes
checking for main() in -lcblas... yes
checking for main() in -latlas... yes
checking for atlas/cblas.h... no
checking for cblas.h... yes
checking for clapack.h... yes
checking for clapack_dgetrf() in cblas.h,clapack.h... no
checking for clapack_dgetri() in cblas.h,clapack.h... no
checking for dgesvd_() in clapack.h... yes
checking for cblas_dgemm() in cblas.h... yes
creating nmatrix_config.h
creating Makefile

How would I remove clapack.h from the installation path? I've already unset C_INCLUDE_PATH and CPLUS_INCLUDE_PATH.

Edit: I think I have it...updates soon.

andrewcsmith · 2014-04-08T19:21:18Z

Fixed it. I had a symlink from /usr/local/include/clapack.h to the LAPACK header in the Accelerate framework, and that was interrupting things.

However, it's bringing me back to why I brewed my own to begin with. I can't find the inverse of the matrix without having LAPACK installed. (Not to hijack this thread, but...) do you know if there's a way to allow #inverse to work?

2.0.0-p451 :027 > pp n

[
  [2, 3]   [5, 4] ]

2.0.0-p451 :028 > pp n.invert

[
  [(-4/7), (3/7)]   [ (5/7), (2/7)] ]

@lazywei, have you had any further luck?

translunar · 2014-04-08T19:22:36Z

There should be. It works on my machine. But it didn't always, and I honestly don't know what I did to make it work. Sorry. I know that's not very helpful. I'd be curious to hear what you make work.

I will say that originally, only cblas.h was in the search path, and clapack.h was not. I accomplished this, most likely, through symbolic links.

andrewcsmith · 2014-04-08T19:26:58Z

Okay, I'll work on it — perhaps add an inversion fix section to the wiki as well. Is there a related bug tracking for this? To me, it seems like a bug, or at least a lack of a feature.

translunar · 2014-04-08T19:33:22Z

Unfortunately, it's not really an NMatrix bug. It's a problem with your environment. There are so many potential environment problems that it's kind of outside the scope of our tracker. With that said, I think we'd like to document the most common cases so people can fix them.

Maybe I could add a "document" tag?

andrewcsmith · 2014-04-08T19:45:23Z

Yeah, that would be great. So, even if I follow this section in the wiki, I should still be able to invert?

We want to include cblas.h but not clapack.h. Both are in /System/Library/Frameworks/Accelerate.framework/Versions/Current/Frameworks/vecLib.framework/Versions/Current/Headers/, so we can't just set C_INCLUDE_PATH and CPLUS_INCLUDE_PATH. Instead, create a symlink:

That's the part I don't get — does this step actually remove LAPACK, and therefore remove inversion, or is the environment supposed to revert to something else?

I hoped that $ gem install pkg/nmatrix-0.1.0.rc4.gem -- --with-lapacklib would add the inversion, but that doesn't seem to work. I'll keep playing around with it, of course, and see if I can get my environment fixed.

translunar · 2014-04-08T19:49:59Z

Well, it's worth noting that there's a distinction between clapack and lapack. I believe clapack is needed for inversion. lapack is only for geev, gesv, and the other eigenvalue decomposition stuff.

andrewcsmith · 2014-04-12T00:37:04Z

Okay, I'm not just what happened exactly, but I was getting some pretty gnarly errors in the spec. I reinstalled Ruby with the command

rvm reinstall 2.0.0-p451 --with-gcc=/usr/local/bin/gcc

...pointing rvm to the correct Homebrew version of gcc (4.9, fwiw) and now my specs all pass. So it works!

The only downside is that there's a huge list of specs that are still "pending" due to not being supported without CLAPACK or LAPACK linked.

translunar · 2014-05-23T05:32:22Z

Assuming this issue is safe to close now. Re-open if not.

translunar added the bug label Mar 18, 2014

translunar added documentation and removed bug labels Apr 8, 2014

andrewcsmith mentioned this issue May 14, 2014

Fixed issues with the #have_func calls and the "unused variable" warnings #238

Merged

translunar closed this as completed May 23, 2014

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Install on Mavericks (Mac OS X 10.9) #209

Install on Mavericks (Mac OS X 10.9) #209

lazywei commented Mar 16, 2014

translunar commented Mar 18, 2014

lazywei commented Mar 18, 2014

translunar commented Mar 18, 2014

thisMagpie commented Apr 5, 2014

translunar commented Apr 5, 2014

andrewcsmith commented Apr 8, 2014

translunar commented Apr 8, 2014

andrewcsmith commented Apr 8, 2014

translunar commented Apr 8, 2014

andrewcsmith commented Apr 8, 2014

translunar commented Apr 8, 2014

andrewcsmith commented Apr 8, 2014

translunar commented Apr 8, 2014

andrewcsmith commented Apr 8, 2014

translunar commented Apr 8, 2014

translunar commented Apr 8, 2014

andrewcsmith commented Apr 8, 2014

andrewcsmith commented Apr 8, 2014

translunar commented Apr 8, 2014

andrewcsmith commented Apr 8, 2014

translunar commented Apr 8, 2014

andrewcsmith commented Apr 8, 2014

translunar commented Apr 8, 2014

andrewcsmith commented Apr 12, 2014

translunar commented May 23, 2014

Install on Mavericks (Mac OS X 10.9) #209

Install on Mavericks (Mac OS X 10.9) #209

Comments

lazywei commented Mar 16, 2014

translunar commented Mar 18, 2014

lazywei commented Mar 18, 2014

translunar commented Mar 18, 2014

thisMagpie commented Apr 5, 2014

translunar commented Apr 5, 2014

andrewcsmith commented Apr 8, 2014

translunar commented Apr 8, 2014

andrewcsmith commented Apr 8, 2014

translunar commented Apr 8, 2014

andrewcsmith commented Apr 8, 2014

translunar commented Apr 8, 2014

andrewcsmith commented Apr 8, 2014

translunar commented Apr 8, 2014

andrewcsmith commented Apr 8, 2014

translunar commented Apr 8, 2014

translunar commented Apr 8, 2014

andrewcsmith commented Apr 8, 2014

andrewcsmith commented Apr 8, 2014

translunar commented Apr 8, 2014

andrewcsmith commented Apr 8, 2014

translunar commented Apr 8, 2014

andrewcsmith commented Apr 8, 2014

translunar commented Apr 8, 2014

andrewcsmith commented Apr 12, 2014

translunar commented May 23, 2014