gfortran warnings on macOS #3
Comments
|
Thanks! By chance, did you replace the files in that shipped with R from CRAN with Apple's BLAS/LAPACK manually? I ask because that seems reasonable, but I otherwise can't explain the behavior. |
|
Yeah, I did it manually. I think I basically followed these instructions https://cran.r-project.org/bin/macosx/RMacOSX-FAQ.html#Which-BLAS-is-used-and-how-can-it-be-changed_003f or these http://statistics.berkeley.edu/computing/blas |
|
I think it should be working now. Thanks again! |
|
Confirm that's tidied up almost all of them. A few of > install_github('wrathematics/spm')
Using GitHub PAT from envvar GITHUB_PAT
Downloading GitHub repo wrathematics/spm@master
from URL https://api.github.com/repos/wrathematics/spm/zipball/master
Installing spm
'/Library/Frameworks/R.framework/Resources/bin/R' --no-site-file \
--no-environ --no-save --no-restore --quiet CMD INSTALL \
'/private/var/folders/f1/6pjy5xbn0_9_7xwq6l7fj2yc0000gn/T/RtmpcdQsLU/devtools2dd46b75800/wrathematics-spm-23ebc19' \
--library='/Library/Frameworks/R.framework/Versions/3.4/Resources/library' \
--install-tests
* installing *source* package ‘spm’ ...
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking for gcc option to accept ISO C99... none needed
BLAS_LIBS=-L/Library/Frameworks/R.framework/Resources/lib -lRblas
LAPACK_LIBS=-L/Library/Frameworks/R.framework/Resources/lib -lRlapack
checking for echo... yes
checking for grep... yes
checking for sgemm_ in -lRblas... no
checking for sgetrf_ in -lRlapack... no
checking for g77... no
checking for xlf... no
checking for f77... no
checking for frt... no
checking for pgf77... no
checking for cf77... no
checking for fort77... no
checking for fl32... no
checking for af77... no
checking for xlf90... no
checking for f90... no
checking for pgf90... no
checking for pghpf... no
checking for epcf90... no
checking for gfortran... gfortran
checking whether we are using the GNU Fortran 77 compiler... yes
checking whether gfortran accepts -g... yes
Using internal single precision blas
Using internal single precision lapack
configure: creating ./config.status
config.status: creating src/Makevars
** libs
gfortran -fopenmp -fPIC -g -O2 -c lapack/slapack1.f -o lapack/slapack1.o
gfortran -fopenmp -fPIC -g -O2 -c lapack/slapack2.f -o lapack/slapack2.o
gfortran -fopenmp -fPIC -g -O2 -c lapack/slapack3.f -o lapack/slapack3.o
gfortran -fopenmp -fPIC -g -O2 -c lapack/slapack4.f -o lapack/slapack4.o
gfortran -fopenmp -fPIC -g -O2 -c lapack/slamchf77.f -o lapack/slamchf77.o
gfortran -fopenmp -fPIC -g -O2 -c lapack/ilas.f -o lapack/ilas.o
gfortran -fopenmp -fPIC -g -O2 -c lapack/sblas.f -o lapack/sblas.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I/usr/local/include -fopenmp -fPIC -Wall -g -O2 -c bracket.c -o bracket.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I/usr/local/include -fopenmp -fPIC -Wall -g -O2 -c chol.c -o chol.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I/usr/local/include -fopenmp -fPIC -Wall -g -O2 -c converters.c -o converters.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I/usr/local/include -fopenmp -fPIC -Wall -g -O2 -c crossprod.c -o crossprod.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I/usr/local/include -fopenmp -fPIC -Wall -g -O2 -c diag.c -o diag.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I/usr/local/include -fopenmp -fPIC -Wall -g -O2 -c dims.c -o dims.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I/usr/local/include -fopenmp -fPIC -Wall -g -O2 -c isSymmetric.c -o isSymmetric.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I/usr/local/include -fopenmp -fPIC -Wall -g -O2 -c matmult.c -o matmult.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I/usr/local/include -fopenmp -fPIC -Wall -g -O2 -c numbytes.c -o numbytes.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I/usr/local/include -fopenmp -fPIC -Wall -g -O2 -c rand.c -o rand.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I/usr/local/include -fopenmp -fPIC -Wall -g -O2 -c solve.c -o solve.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I/usr/local/include -fopenmp -fPIC -Wall -g -O2 -c xpose.c -o xpose.o
/usr/local/clang4/bin/clang -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/usr/local/clang4/lib -o spm.so lapack/slapack1.o lapack/slapack2.o lapack/slapack3.o lapack/slapack4.o lapack/slamchf77.o lapack/ilas.o lapack/sblas.o bracket.o chol.o converters.o crossprod.o diag.o dims.o isSymmetric.o matmult.o numbytes.o rand.o solve.o xpose.o -L/Library/Frameworks/R.framework/Resources/lib -lRlapack -L/Library/Frameworks/R.framework/Resources/lib -lRblas -L/usr/local/gfortran/lib/gcc/x86_64-apple-darwin15/6.1.0 -L/usr/local/gfortran/lib -lgfortran -lquadmath -lm -fopenmp -fopenmp -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
ld: warning: directory not found for option '-L/usr/local/gfortran/lib/gcc/x86_64-apple-darwin15/6.1.0'
ld: warning: could not create compact unwind for _spbtrf_: stack subq instruction is too different from dwarf stack size
ld: warning: could not create compact unwind for _shseqr_: stack subq instruction is too different from dwarf stack size
ld: warning: could not create compact unwind for _sgbtrf_: stack subq instruction is too different from dwarf stack size
installing to /Library/Frameworks/R.framework/Versions/3.4/Resources/library/spm/libs
** R
** tests
** byte-compile and prepare package for lazy loading
Creating a generic function for ‘crossprod’ from package ‘base’ in package ‘spm’
Creating a generic function for ‘tcrossprod’ from package ‘base’ in package ‘spm’
Creating a generic function for ‘diag’ from package ‘base’ in package ‘spm’
Creating a generic function for ‘nrow’ from package ‘base’ in package ‘spm’
Creating a generic function for ‘ncol’ from package ‘base’ in package ‘spm’
Creating a generic function for ‘object.size’ from package ‘utils’ in package ‘spm’
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded
* DONE (spm) |
|
So it's not finding the single precision symbols in your libraries for some reason. These are the relevant lines: This doesn't make sense to me. The BLAS/LAPACK libs it's detecting are: Now I know for a fact that you have the necessary functions, because you were able to build the package before I added all of those fortran files as a backup (you shouldn't be building them, given your setup). I've tested something similar on a local Linux setup and things worked fine, so I'm not sure what the deal is. But I'd bet that if you run: that you will get a match. It's possible that Macs don't respect |
|
Turns out |
|
Ah no, that's my bad. I forgot macs use So the thing to try would be: If that doesn't have a match, then I'm going to be quite confused how you ever installed it in the first place 😅 |
|
Ugh, turns out this is probably because I misremembered how I set up this machine. I repeated the steps in http://statistics.berkeley.edu/computing/blas and now get: So now it's actually using Accelerate's libBLAS whereas it wasn't before (99% sure, anyway). I misremembered the setup of this machine. And when installing spm (full output at end): So yay, spm is picking up the Accelerate BLAS. But nothing for LAPACK? Also, I can't find Sorry for the noise on this issue - it's almost certainly caused by the bizarro world set up on my Mac. Out of interest, have you been able to test on a bog-standard macOS set up? > install_github("wrathematics/spm", force = TRUE)
Using GitHub PAT from envvar GITHUB_PAT
Downloading GitHub repo wrathematics/spm@master
from URL https://api.github.com/repos/wrathematics/spm/zipball/master
Installing spm
'/Library/Frameworks/R.framework/Resources/bin/R' --no-site-file \
--no-environ --no-save --no-restore --quiet CMD INSTALL \
'/private/var/folders/f1/6pjy5xbn0_9_7xwq6l7fj2yc0000gn/T/RtmpWpd85I/devtoolsba725900e9d/wrathematics-spm-23ebc19' \
--library='/Library/Frameworks/R.framework/Versions/3.4/Resources/library' \
--install-tests
* installing *source* package ‘spm’ ...
checking for gcc... gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking for gcc option to accept ISO C99... none needed
BLAS_LIBS=-L/Library/Frameworks/R.framework/Resources/lib -lRblas
LAPACK_LIBS=-L/Library/Frameworks/R.framework/Resources/lib -lRlapack
checking for echo... yes
checking for grep... yes
checking for sgemm_ in -lRblas... yes
checking for sgetrf_ in -lRlapack... no
checking for g77... no
checking for xlf... no
checking for f77... no
checking for frt... no
checking for pgf77... no
checking for cf77... no
checking for fort77... no
checking for fl32... no
checking for af77... no
checking for xlf90... no
checking for f90... no
checking for pgf90... no
checking for pghpf... no
checking for epcf90... no
checking for gfortran... gfortran
checking whether we are using the GNU Fortran 77 compiler... yes
checking whether gfortran accepts -g... yes
Using system single precision blas
Using internal single precision lapack
configure: creating ./config.status
config.status: creating src/Makevars
** libs
gfortran -fopenmp -fPIC -g -O2 -c lapack/slapack1.f -o lapack/slapack1.o
gfortran -fopenmp -fPIC -g -O2 -c lapack/slapack2.f -o lapack/slapack2.o
gfortran -fopenmp -fPIC -g -O2 -c lapack/slapack3.f -o lapack/slapack3.o
gfortran -fopenmp -fPIC -g -O2 -c lapack/slapack4.f -o lapack/slapack4.o
gfortran -fopenmp -fPIC -g -O2 -c lapack/slamchf77.f -o lapack/slamchf77.o
gfortran -fopenmp -fPIC -g -O2 -c lapack/ilas.f -o lapack/ilas.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I/usr/local/include -fopenmp -fPIC -Wall -g -O2 -c bracket.c -o bracket.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I/usr/local/include -fopenmp -fPIC -Wall -g -O2 -c chol.c -o chol.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I/usr/local/include -fopenmp -fPIC -Wall -g -O2 -c converters.c -o converters.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I/usr/local/include -fopenmp -fPIC -Wall -g -O2 -c crossprod.c -o crossprod.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I/usr/local/include -fopenmp -fPIC -Wall -g -O2 -c diag.c -o diag.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I/usr/local/include -fopenmp -fPIC -Wall -g -O2 -c dims.c -o dims.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I/usr/local/include -fopenmp -fPIC -Wall -g -O2 -c isSymmetric.c -o isSymmetric.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I/usr/local/include -fopenmp -fPIC -Wall -g -O2 -c matmult.c -o matmult.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I/usr/local/include -fopenmp -fPIC -Wall -g -O2 -c numbytes.c -o numbytes.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I/usr/local/include -fopenmp -fPIC -Wall -g -O2 -c rand.c -o rand.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I/usr/local/include -fopenmp -fPIC -Wall -g -O2 -c solve.c -o solve.o
/usr/local/clang4/bin/clang -I/Library/Frameworks/R.framework/Resources/include -DNDEBUG -I/usr/local/include -fopenmp -fPIC -Wall -g -O2 -c xpose.c -o xpose.o
/usr/local/clang4/bin/clang -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/usr/local/clang4/lib -o spm.so lapack/slapack1.o lapack/slapack2.o lapack/slapack3.o lapack/slapack4.o lapack/slamchf77.o lapack/ilas.o bracket.o chol.o converters.o crossprod.o diag.o dims.o isSymmetric.o matmult.o numbytes.o rand.o solve.o xpose.o -L/Library/Frameworks/R.framework/Resources/lib -lRlapack -L/Library/Frameworks/R.framework/Resources/lib -lRblas -L/usr/local/gfortran/lib/gcc/x86_64-apple-darwin15/6.1.0 -L/usr/local/gfortran/lib -lgfortran -lquadmath -lm -fopenmp -fopenmp -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
ld: warning: directory not found for option '-L/usr/local/gfortran/lib/gcc/x86_64-apple-darwin15/6.1.0'
ld: warning: could not create compact unwind for _spbtrf_: stack subq instruction is too different from dwarf stack size
ld: warning: could not create compact unwind for _shseqr_: stack subq instruction is too different from dwarf stack size
ld: warning: could not create compact unwind for _sgbtrf_: stack subq instruction is too different from dwarf stack size
installing to /Library/Frameworks/R.framework/Versions/3.4/Resources/library/spm/libs
** R
** tests
** byte-compile and prepare package for lazy loading
Creating a generic function for ‘crossprod’ from package ‘base’ in package ‘spm’
Creating a generic function for ‘tcrossprod’ from package ‘base’ in package ‘spm’
Creating a generic function for ‘diag’ from package ‘base’ in package ‘spm’
Creating a generic function for ‘nrow’ from package ‘base’ in package ‘spm’
Creating a generic function for ‘ncol’ from package ‘base’ in package ‘spm’
Creating a generic function for ‘object.size’ from package ‘utils’ in package ‘spm’
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded
* DONE (spm) |
|
The internet seems to be suggesting using Apparently some BLAS vendors put the LAPACK symbols in the BLAS library itself (which is news to me!). OpenBLAS does this, and it's possible that Apple's Accelerate is doing this. This may explain how you were able to build the package before I added the ability to build them from source if they were missing (unless you also changed So it might be interesting to test: I think you will see s, d, c, z matches for the first one, but only d and z matches for the second. If not, then this really is a mystery 😅 This has been extremely helpful to me, so I really appreciate all the help and patience! |
|
So no hits in # The old (default) BLAS I was using
Peter$ nm -gU /Library/Frameworks/R.framework/Resources/lib/libRblas.dylib.bak | grep gesdd
# The Accelerate BLAS that I am using
Peter$ nm -gU /Library/Frameworks/R.framework/Resources/lib/libRblas.dylib | grep gesdd
# The unchanged LAPACK
Peter$ nm -gU /Library/Frameworks/R.framework/Resources/lib/libRlapack.dylib | grep gesdd
00000000000f7150 T _dgesdd_
000000000018c0e0 T _zgesdd_I can post the full list of routines if that is helpful, but I just noticed I have # The Accelerate LAPACK
Peter$ nm -gU /System/Library/Frameworks/Accelerate.framework/Versions/Current/Frameworks/vecLib.framework/Versions/Current/libLAPACK.dylib | grep gesdd
00000000000278fb T _cgesdd
00000000000278fb T _cgesdd_
0000000000129a55 T _dgesdd
0000000000129a55 T _dgesdd_
0000000000230657 T _sgesdd
0000000000230657 T _sgesdd_
000000000032af32 T _zgesdd
000000000032af32 T _zgesdd_I tried to explicitly use it via the same trick I used for switching out the BLAS dylib: ln -s /System/Library/Frameworks/Accelerate.framework/Versions/Current/Frameworks/vecLib.framework/Versions/Current/libLAPACK.dylib libRlapack.dylibbut bad things happened on starting R Error: package or namespace load failed for ‘stats’ in dyn.load(file, DLLpath = DLLpath, ...):
unable to load shared object '/Library/Frameworks/R.framework/Versions/3.4/Resources/library/stats/libs/stats.so':
dlopen(/Library/Frameworks/R.framework/Versions/3.4/Resources/library/stats/libs/stats.so, 6): Library not loaded: /Library/Frameworks/R.framework/Versions/3.4/Resources/lib/libRlapack.dylib
Referenced from: /Library/Frameworks/R.framework/Versions/3.4/Resources/library/stats/libs/stats.so
Reason: Incompatible library version: stats.so requires version 3.4.0 or later, but libLAPACK.dylib provides version 1.0.0
During startup - Warning message:
package ‘stats’ in options("defaultPackages") was not found |
|
I have same issue with installation on OS X. Accelerate BLAS is simlinked to libRblas, Lapack is default R's lapack (because linkage of accelerate lapack produce "stats.so requires version 3.4.0 or later, but libLAPACK.dylib provides version 1.0.0") Here is full log. Not sure why configure decided to use libRlapack:
|
|
At the moment, the configure script looks for What's stranger is that your compiler and libc are missing a bunch of C99 functions. |
|
I think in that sense configure works correctly - on my system Does following string mean that package is trying use lapack from package lapack subdir?
For me this looks like it didn't find single precision system lapack, but still tries to link to it (but i'm noob in these configure things)..
Let me know if I can help somehow. |
|
Yes, the message means it's using the package's internal single precision LAPACK. It's not well worded. I'll try to fix it. The package ships the float BLAS and LAPACK source since many will only have the double precision symbols. There's obviously a huge advantage in using optimized BLAS (not the ones in the package, which are the "reference" implementations from netlib). For LAPACK it may not matter much from a run time perspective. However, the compilation time will be significantly longer if it has to compile LAPACK, so it's still desirable to find those symbols if they're there. So currently the configure script looks at the BLAS library linked with R for single precision BLAS symbols, and it looks at the LAPACK library and the BLAS library for single precision LAPACK symbols (since Accelerate and OpenBLAS include LAPACK symbols in the "BLAS" library). This works on my Linux box with OpenBLAS, but Accelerate is still a bit of a mystery to me. |
|
I just reinstalled R with brew: Installation also works with R from CRAN, but there are several warnings at the end:
Don't know what they are about. |
Following from #2 (comment), here's the full output. I'm using Apple's Accelerate BLAS (without fully understanding why, except 'it'll be faster')
The text was updated successfully, but these errors were encountered: