Windows installation instructions

[remcopeereboom]

The documentation does not have any installation instructions for windows. It would be nice if there were a guide that walked users through getting matrix-lapacke and matrix-atlas to run. The GNU toolchain on Windows is something of a nightmare if you are not used to Linux commands and the installation Windows instructions on the Lapack and OpenBLAS websites are difficult and horribly outdated.

Also, is there a way to dynamically bind to the libraries (e.g. through ffi) or do we need to build statically. Dynamic bindings would be easier because pre-compiled binaries are available (obviously not optimized for a users system, but a nice no-need-to-think-just-install fall-back).

[translunar]

@gau27 @preetpalS @ThRabbit Are any of you all able to help with this?

[preetpalS]

@remcopeereboom How are you using Ruby (CRuby) on Windows. If you're not using oneclick/rubyinstaller2, you should look into it. That packaged ruby installer installs ruby alongside msys2 (which is reliable; the official Git build on Windows uses it I believe) which includes the Arch Linux package manager pacman which allows you to easily install almost all Linux software on Windows including lapack and openblas.

For example to install lapack and openblas on Windows, using pacman use the following commands (assuming you are using the 64-bit version of Ruby):

pacman -S mingw-w64-x86_64-openblas mingw-w64-x86_64-lapack

Afterwards you technically are supposed to be able install nmatrix-lapacke normally on Windows like you would with other gems but this is not the case today as you will run into the linking errors on Windows (or at least I did with Ruby 2.4.1p111 (2017-03-22 revision 58053)). I'll most likely look into this issue further but I cannot make any promises (either timelines or actually being able to fix the actual issues). Note that the nmatrix gem (version 0.2.3) installs on Windows and works fine.

Also you might want to look into the native Linux layer on Windows (I forget what it's called but it allows you to run Ubuntu binaries directly on Windows; it also includes the Ubuntu package manager so you can install everything like you would on Ubuntu).

[remcopeereboom]

@preetpalS Thanks for the reply. I'm using MRI with the oneclick installer. At the time my first issue was finding out what pacman was, because that's not obvious to someone used to Windows. I did manage to get there in the end, but I assumed the error I got was because I somehow failed to install lapacke or failed to install it in the right place since ruby can't seem to find it when installing native extensions:

ERROR:  Error installing nmatrix-lapacke:
        ERROR: Failed to build gem native extension.

    current directory: C:/Ruby24-x64/lib/ruby/gems/2.4.0/gems/nmatrix-lapacke-0.2.3/ext/nmatrix_lapacke
C:/Ruby24-x64/bin/ruby.exe -r ./siteconf20171123-8688-1akevv.rb extconf.rb
using C++ standard... c++11
g++ reports version... Built
checking for rb_array_const_ptr() in ruby.h... yes
checking for FIX_CONST_VALUE_PTR in ruby.h... yes
checking for RARRAY_CONST_PTR in ruby.h... yes
checking for RARRAY_AREF in ruby.h... yes
checking for -llapack... no
creating nmatrix_lapacke_config.h
creating Makefile

current directory: C:/Ruby24-x64/lib/ruby/gems/2.4.0/gems/nmatrix-lapacke-0.2.3/ext/nmatrix_lapacke
make "DESTDIR=" clean

current directory: C:/Ruby24-x64/lib/ruby/gems/2.4.0/gems/nmatrix-lapacke-0.2.3/ext/nmatrix_lapacke
make "DESTDIR="
generating nmatrix_lapacke-x64-mingw32.def
compiling nmatrix_lapacke.cpp
compiling math_lapacke.cpp
compiling lapacke.cpp
linking shared-object nmatrix_lapacke.so
C:/msys64/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/6.2.0/../../../../x86_64-w64-mingw32/bin/ld.exe: cannot find -llapack
collect2.exe: error: ld returned 1 exit status
make: *** [Makefile:259: nmatrix_lapacke.so] Error 1

make failed, exit code 2

Gem files will remain installed in C:/Ruby24-x64/lib/ruby/gems/2.4.0/gems/nmatrix-lapacke-0.2.3 for inspection.
Results logged to C:/Ruby24-x64/lib/ruby/gems/2.4.0/extensions/x64-mingw32/2.4.0/nmatrix-lapacke-0.2.3/gem_make.out
Temporarily enhancing PATH for MSYS/MINGW..

[preetpalS]

But are using rubyinstaller2? I am using one of the releases from Github. It should have setup the path for you, so you shouldn't be running into that particular issue. When I attempted the installation, I ran into a different set of errors (linking errors).

[preetpalS]

@remcopeereboom Basically to simplify things, during the installation of the gem on your computer, your Ruby installation's compiler toolchain cannot find lapack. If you installed Ruby with rubyinstaller2, and installed lapack with pacman, you should not have run into that particular issue. Although, even if you didn't run into that issue, chances are (based on what happened when I tried to install the nmatrix-lapacke gem) that you still would not be able to install the gem because of other issues.

@MohawkJohn, @v0dro I have attached the compilation error log files from the attempted gem installation on Windows 10 to this issue. If you have any insight into how to resolve these issues that would be helpful in moving forward.

mkmf.log
gem_make.out.txt

[remcopeereboom]

@preetpalS

But are using rubyinstaller2? I am using one of the releases from Github. It should have setup the path for you, so you shouldn't be running into that particular issue. When I attempted the installation, I ran into a different set of errors (linking errors).

Thanks again for taking the time to respond to this issue. Yes, as I mentioned, I am using rubyinstaller2, and I did install lapack with pacman from the msys2 installed by the ruby installer, nevertheless, I still got the error shown when gem installing.

[preetpalS]

I looked more into the issue. The first issue that should be resolved in the gem installation process (on my computer where Lapack is found by the toolchain) on Windows is why the following error happens:

linking shared-object nmatrix_lapacke.so
math_lapacke.o:math_lapacke.cpp:(.text+0xd2): undefined reference to `rubyval_to_cval'

From my understanding rubyval_to_cval is a function defined here:

nmatrix/ext/nmatrix/data/data.cpp

Line 198 in 7f3fa5d

void rubyval_to_cval(VALUE val, nm::dtype_t dtype, void* loc) {

This would mean that the compilation process for the nmatrix-lapacke gem on Windows (using RubyInstaller2) is running into issues not even related to lapack.

Maybe one of the developers from RubyInstaller2 would know how to fix this. @larskanis

[larskanis]

The current mechanism for loading nmatrix plugins works, because some symbols (like rubyval_to_cval) are kept unresolved while linking and the nmatrix.so file is always loaded before nmatrix_lapack.so and the dynamic loader of Linux and OSX is smart enough to link the unresolved symbols from nmatrix_lapack.so to nmatrix.so at runtime.

Unfortunately this doesn't work on Windows, because unresolved symbols are not allowed there and can not be resolved at runtime. There are two ways to overcome this issue:

1. Link data.cpp static into nmatrix_lapack.so: The extconf.rb run could copy the required files into the lapack source directory, so that they get compiled in. This avoids any unresolved symbols, but actually loads the code of data.cpp twice (once in nmatrix_lapack.so and another one nmatrix.so).

2. Dynamic link to nmatrix.so explicitly: This requires that rubyval_to_cval is marked as an export function and that the dependency to nmatrix.so is made explicit (by LDFLAGS=-l:nmatrix.so or so).