Mac wheel library linking issue

[bnaul]

I've been using glmnet==2.2.1 installed from Mac wheels with gcc==9.3.0 with no issues. But when my colleagues who didn't have gcc installed yet tried to go through the setup, they ran into the following:

python -c "import glmnet"

ImportError: dlopen(/Users/rockwellweiner/model/.venv/lib/python3.7/site-packages/_glmnet.cpython-37m-darwin.so, 2): Library not loaded: /usr/local/opt/gcc/lib/gcc/9/libgfortran.5.dylib Referenced from: /Users/rockwellweiner/model/.venv/lib/python3.7/site-packages/_glmnet.cpython-37m-darwin.so Reason: image not found

Symlinking /usr/local/opt/gcc/lib/gcc/10 to /usr/local/opt/gcc/lib/gcc/9 seems to do the trick but is obviously not ideal; is there maybe a change to setup.py or the wheel build script that would support both?

[jacksonllee]

Hello, as far as I can tell, there have been no changes within setup.py for compiling Fortran code in the past several years (the other changes I made a week ago for the v2.2.1 release don't seem to be related):

https://github.com/civisanalytics/python-glmnet/blame/813c06f5fcc9604d8e445bd4992f53c4855cc7cb/setup.py

As for building the macosx wheels, I created them by running the python setup.py bdist_wheel command, with python being each of the Python versions we now support.

To improve the pip-install experience for Mac users, if having gcc installed would solve the issue, then perhaps we could more explicitly recommend brew install gcc before running pip install glmnet in the documentation (in the current readme, we mention brew install gcc only in the paragraph about installing python-glmnet from source).

[bnaul]

Thanks for the quick response @jacksonllee. The issue is that currently brew install gcc && pip install glmnet && python -c "import glmnet" does not work; I happened to have gcc==9.3 installed already, but now brew gives you gcc==10.1, and the wheel is looking in the wrong place (hardcoded to gcc/9.

I'm guessing it's looking for 9 because that's what you happened to have locally as well (and probably that's been the Homebrew version since the commit you linked to); I'm afraid I don't know enough to say how the explicit 9 is making it into the expected library path though...

[jacksonllee]

To confirm, I've just checked my Mac, and indeed it's on gcc 9.3. I also don't quite see how, if at all, the gcc version is hard-coded for building the wheels. One thing I can think of off the top of my head is that the Fortran compilation is done by the packaging tools imported from numpy which may be relevant to the issue at hand, but I'm not sure. We may also look at how other more established projects with Fortran code (scipy, etc.) build wheels and then we can emulate (the current set-up here is admittedly a couple years old and is probably due for maintenance). When our engineering resources allow it, we'll be able to look into this ticket more.

[liyi-1989]

I got similar error on my Mac. I only have gcc 10 but no 9. Basically, I install gcc 9 with brew, and link the installed folder to the one mentioned in the glmnet error:

brew install gcc@9 
#  installed in /usr/local/Cellar/gcc@9/9.3.0/
ln -s /usr/local/Cellar/gcc@9/9.3.0/lib/gcc/9 /usr/local/opt/gcc/lib/gcc/9

[JacobHayes]

Any chance the version could be bumped and new wheels made for gcc 10 (+ python 3.9)?

[hugovergnes]

I got similar error on my Mac. I only have gcc 10 but no 9. Basically, I install gcc 9 with brew, and link the installed folder to the one mentioned in the glmnet error:

brew install gcc@9 
#  installed in /usr/local/Cellar/gcc@9/9.3.0/
ln -s /usr/local/Cellar/gcc@9/9.3.0/lib/gcc/9 /usr/local/opt/gcc/lib/gcc/9

For now, this solution work on mac (Not M1). Thanks !

[amirhessam88]

Seems like the repo is kinda dead and no one is maintaining it!