Vowpalwabbit python wrappers do not build or crashes on OSX with Anaconda

[peterychang]

There is a known issue on OSX 10.14+ with the currently deployed version of python on Anaconda.

The problem is as follows:

• Python requires linking binaries (eg: VW) to also link against the same c++ library it did.
• Anaconda builds python against an older version of OSX and links against libstdc++
• libstdc++ was deprecated by Apple/Clang several years ago. And as of OSX 10.14, the library was removed entirely.

As a result, VW will typically fail to compile or install on the latest version of OSX. Using compiler options to force the use of libc++ will cause python to crash.

At the moment, the only resolutions available to users are to either uninstall Anaconda and use Homebrew or to copy the missing libraries from an older version of xcode.

We are currently exploring ways to resolve these compatibility issues, but there does not appear to be a quick or easy solution to the problem.

If this is a critical issue for you, please feel free to reach out to us so we can see if there is an individual solution that may work.

We also welcome anybody who would like to contribute to the resolution of this issue.

[peterychang]

For users of Homebrew having trouble installing VW from pip:

clang: error: no input files and boost-cmake errors have been resolved as of #2097
These fixes will be available via pip from our next release.
For anybody unable to wait for the next release, please install the package from source.

[scnullois]

Having same issue with pip on OSX. When is the fixed version going to be available on pip?

    [100%] Built target pylibvw
    building 'pylibvw' extension
    clang -bundle -undefined dynamic_lookup -L/usr/local/opt/readline/lib -L/usr/local/opt/readline/lib -L/usr/local/opt/openssl/lib -L/Users/scgalois/.pyenv/versions/3.7.0/lib -L/usr/local/opt/openssl/lib -L/usr/local/opt/readline/lib -L/usr/local/opt/readline/lib -L/usr/local/opt/openssl/lib -L/Users/scgalois/.pyenv/versions/3.7.0/lib -L/usr/local/opt/openssl/lib -o build/lib.macosx-10.14-x86_64-3.7/pylibvw.so
    clang: error: no input files
    error: command 'clang' failed with exit status 1
    ----------------------------------------
ERROR: Command errored out with exit status 1: /Users/scgalois/.pyenv/versions/3.7.0/envs/gc_all/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/zx/r2fggd2x6x36_vbyqv6jxp9c0000gn/T/pip-install-5fn40s7t/vowpalwabbit/setup.py'"'"'; __file__='"'"'/private/var/folders/zx/r2fggd2x6x36_vbyqv6jxp9c0000gn/T/pip-install-5fn40s7t/vowpalwabbit/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/zx/r2fggd2x6x36_vbyqv6jxp9c0000gn/T/pip-record-z_peocph/install-record.txt --single-version-externally-managed --compile --install-headers /Users/scgalois/.pyenv/versions/3.7.0/envs/gc_all/include/site/python3.7/vowpalwabbit Check the logs for full command output.```

[scnullois]

Also if it will be more than a few days, are there instructions for installing package from source please?

I tried to clone the repo and then ran the instructions from the issue with the fix but I still get following error:

(gc_all) LOCAL ➜  vowpal_wabbit git:(master) pip install --global-option="--enable-boost-cmake" --global-option="--cmake-options=--verbose;-DCMAKE_OPTION=ON" vowpalwabbit
/Users/scgalois/.pyenv/versions/3.7.0/envs/gc_all/lib/python3.7/site-packages/pip/_internal/commands/install.py:283: UserWarning: Disabling all use of wheels due to the use of --build-options / --global-options / --install-options.
  cmdoptions.check_install_build_global(options)
Collecting vowpalwabbit
  Using cached https://files.pythonhosted.org/packages/f4/09/50efc9630809fdd19056d2fff8e7b8e374063002b5a70b016854fe5e8492/vowpalwabbit-8.7.0.post1.tar.gz
Installing collected packages: vowpalwabbit
    Running setup.py install for vowpalwabbit ... error
    ERROR: Command errored out with exit status 1:
     command: /Users/scgalois/.pyenv/versions/3.7.0/envs/gc_all/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/zx/r2fggd2x6x36_vbyqv6jxp9c0000gn/T/pip-install-m2l1e7fa/vowpalwabbit/setup.py'"'"'; __file__='"'"'/private/var/folders/zx/r2fggd2x6x36_vbyqv6jxp9c0000gn/T/pip-install-m2l1e7fa/vowpalwabbit/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' --enable-boost-cmake '--cmake-options=--verbose;-DCMAKE_OPTION=ON' install --record /private/var/folders/zx/r2fggd2x6x36_vbyqv6jxp9c0000gn/T/pip-record-v7agrd53/install-record.txt --single-version-externally-managed --compile --install-headers /Users/scgalois/.pyenv/versions/3.7.0/envs/gc_all/include/site/python3.7/vowpalwabbit
         cwd: /private/var/folders/zx/r2fggd2x6x36_vbyqv6jxp9c0000gn/T/pip-install-m2l1e7fa/vowpalwabbit/
    Complete output (6 lines):
    usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
       or: setup.py --help [cmd1 cmd2 ...]
       or: setup.py --help-commands
       or: setup.py cmd --help

    error: option --enable-boost-cmake not recognized
    ----------------------------------------
ERROR: Command errored out with exit status 1: /Users/scgalois/.pyenv/versions/3.7.0/envs/gc_all/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/zx/r2fggd2x6x36_vbyqv6jxp9c0000gn/T/pip-install-m2l1e7fa/vowpalwabbit/setup.py'"'"'; __file__='"'"'/private/var/folders/zx/r2fggd2x6x36_vbyqv6jxp9c0000gn/T/pip-install-m2l1e7fa/vowpalwabbit/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' --enable-boost-cmake '--cmake-options=--verbose;-DCMAKE_OPTION=ON' install --record /private/var/folders/zx/r2fggd2x6x36_vbyqv6jxp9c0000gn/T/pip-record-v7agrd53/install-record.txt --single-version-externally-managed --compile --install-headers /Users/scgalois/.pyenv/versions/3.7.0/envs/gc_all/include/site/python3.7/vowpalwabbit Check the logs for full command output.
(gc_all) LOCAL ➜  vowpal_wabbit git:(master) git pull
Already up to date.
Current branch master is up to date.
(gc_all) LOCAL ➜  vowpal_wabbit git:(master) python setup.py --enable-boost-cmake --cmake-options=--verbose;-DCMAKE_OPTION=ON
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
   or: setup.py --help [cmd1 cmd2 ...]
   or: setup.py --help-commands
   or: setup.py cmd --help

error: no commands supplied
zsh: command not found: -DCMAKE_OPTION=ON

[scnullois]

Ah managed to get it to work by just running python setup.py install from the cloned dir.

[scnullois]

However, now both examples cause segementation fault:

(gc_all) LOCAL ➜  vowpal_wabbit git:(master) ✗ python3
Python 3.7.0 (default, Apr 15 2019, 13:22:51)
[Clang 10.0.1 (clang-1001.0.46.3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from vowpalwabbit import pyvw
>>> vw = pyvw.vw(quiet=False)
Num weight bits = 18
learning rate = 0.5
initial_t = 0
power_t = 0.5
using no cache
Reading datafile =
num sources = 0
>>> ex = vw.example('1 | a b c')
[1]    83919 segmentation fault  python3

(gc_all) LOCAL ➜  vowpal_wabbit git:(master) ✗ python3
Python 3.7.0 (default, Apr 15 2019, 13:22:51)
[Clang 10.0.1 (clang-1001.0.46.3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> from sklearn import datasets
>>> from sklearn.model_selection import train_test_split
>>> from vowpalwabbit.sklearn_vw import VWClassifier
/Users/scgalois/.pyenv/versions/gc_all/lib/python3.7/site-packages/sklearn/externals/joblib/__init__.py:15: DeprecationWarning: sklearn.externals.joblib is deprecated in 0.21 and will be removed in 0.23. Please import this functionality directly from joblib, which can be installed with: pip install joblib. If this warning is raised when loading pickled models, you may need to re-serialize those models with scikit-learn 0.21+.
  warnings.warn(msg, category=DeprecationWarning)
>>>
>>> # generate some data
... X, y = datasets.make_hastie_10_2(n_samples=10000, random_state=1)
>>> X = X.astype(np.float32)
>>>
>>> # split train and test set
... X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=256)
>>>
>>> # build model
... model = VWClassifier()
>>> model.fit(X_train, y_train)
[1]    84429 segmentation fault  python3

[peterychang]

@scgalois I was unable to reproduce your segfault on my setup. Can I get some information from you?

• Which version of OSX are you running on?
• Where did you get python from?
• Version of boost/boost-python3
• Do you have Anaconda installed?

Thanks

[scnullois]

@peterychang thanks for following up. Here are the details:

OSX Mojave 10.14.6 with no anaconda installation (when I type conda in command line says command not found)

(gc_all) LOCAL ➜  vowpal_wabbit git:(master) ✗ which python3
/Users/scgalois/.pyenv/shims/python3
(gc_all) LOCAL ➜  vowpal_wabbit git:(master) ✗ python3 --version
Python 3.7.0

(gc_all) LOCAL ➜  vowpal_wabbit git:(master) ✗ brew search boost
brew==> Formulae
boost ✔                              boost-build                          boost-python ✔                       [email protected]                    [email protected]                           [email protected]
boost-bcp                            boost-mpi                            boost-python3 ✔                      [email protected]                           [email protected]                           xgboost

==> Casks
boostnote                                    focus-booster                                nosqlbooster-for-mongodb                     soundboosterlite                             turbo-boost-switcher
(gc_all) LOCAL ➜  vowpal_wabbit git:(master) ✗ brew info boost boost-python boost-python3
boost: stable 1.71.0 (bottled), HEAD
Collection of portable C++ source libraries
https://www.boost.org/
/usr/local/Cellar/boost/1.71.0 (14,377 files, 608.3MB) *
  Poured from bottle on 2019-10-22 at 13:39:44
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/boost.rb
==> Dependencies
Required: icu4c ✔
==> Options
--HEAD
	Install HEAD version
==> Analytics
install: 41,806 (30 days), 127,389 (90 days), 650,201 (365 days)
install_on_request: 14,825 (30 days), 45,118 (90 days), 210,074 (365 days)
build_error: 0 (30 days)

boost-python: stable 1.71.0 (bottled), HEAD
C++ library for C++/Python2 interoperability
https://www.boost.org/
/usr/local/Cellar/boost-python/1.71.0_1 (460 files, 5.4MB) *
  Poured from bottle on 2019-10-22 at 13:44:48
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/boost-python.rb
==> Dependencies
Required: boost ✔
==> Options
--HEAD
	Install HEAD version
==> Caveats
This formula provides Boost.Python for Python 2. Due to a
collision with boost-python3, the CMake Config files are not
available. Please use -DBoost_NO_BOOST_CMAKE=ON when building
with CMake or switch to Python 3.
==> Analytics
install: 5,433 (30 days), 12,561 (90 days), 55,587 (365 days)
install_on_request: 3,632 (30 days), 8,411 (90 days), 38,916 (365 days)
build_error: 0 (30 days)

boost-python3: stable 1.71.0 (bottled), HEAD
C++ library for C++/Python3 interoperability
https://www.boost.org/
/usr/local/Cellar/boost-python3/1.71.0_1 (472 files, 19.9MB) *
  Poured from bottle on 2019-10-22 at 13:45:24
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/boost-python3.rb
==> Dependencies
Build: numpy ✘
Required: boost ✔, python ✔
==> Options
--HEAD
	Install HEAD version
==> Analytics
install: 2,778 (30 days), 6,485 (90 days), 21,634 (365 days)
install_on_request: 1,575 (30 days), 3,531 (90 days), 12,426 (365 days)
build_error: 0 (30 days)

[peterychang]

@scgalois From your logs, it doesn't look like you're using brew's python.
Specifically, python on brew is 3.7.4_1 compiled against Clang 6.0 (compared to your python 3.7.0 compiled against Clang 10.0.1)

Since your version of python differs from whats on homebrew, its possible the boost python library from homebrew is not binary compatible with the version of python you're running.

If this is the cause, then there are a couple of possible solutions. First, you can switch to python3 from homebrew.
The second option is compiling boost against your current environment and recompiling vowpalwabbit against those libraries

[scnullois]

@peterychang Thanks for the feedback. assuming I clone and compile boost myself, how do I make sure the vw package will link my new version rather than the boost version? Are there any detailed instructions anywhere or do I need to dig into the config scripts to figure it out?

[scnullois]

@peterychang Actually I ended up just changing my version of python and it is working fine now. Thanks for your help. But just for future reference, if there are instructions on building and using older version of boost against pyvw would appreciate it.

[peterychang]

@scgalois If you have a build of boost you want to link against, the following command should force VW to link against it. In this case, you'll want to pay attention to the build logs since I've noticed that the BOOST_ROOT variable is ignored in some cases that I'm still investigating.

python3 setup.py --cmake-options=BOOST_ROOT=/path/to/boost install -f
or via pip (after our next python release) with
pip3 install --global-option="--cmake-options=BOOST_ROOT=/path/to/boost" vowpalwabbit

[elviraux]

Any progress @peterychang ?
I am facing similar issues when I try to install it on macOS Mojave 10.14.6
trying different python versions don't help.

Collecting vowpalwabbit Using cached https://files.pythonhosted.org/packages/d1/5a/9fcd64fd52ad22e2d1821b2ef871e8783c324b37e2103e7ddefa776c2ed7/vowpalwabbit-8.8.0.tar.gz Building wheels for collected packages: vowpalwabbit Building wheel for vowpalwabbit (setup.py) ... error ERROR: Command errored out with exit status 1: command: /Users/someusername/anaconda3/envs/PYTH3610/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/rw/g2c0r4fs0t73v457sqp_h2rh0000gn/T/pip-install-iuanczrt/vowpalwabbit/setup.py'"'"'; __file__='"'"'/private/var/folders/rw/g2c0r4fs0t73v457sqp_h2rh0000gn/T/pip-install-iuanczrt/vowpalwabbit/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /private/var/folders/rw/g2c0r4fs0t73v457sqp_h2rh0000gn/T/pip-wheel-3xwis6xn --python-tag cp36 cwd: /private/var/folders/rw/g2c0r4fs0t73v457sqp_h2rh0000gn/T/pip-install-iuanczrt/vowpalwabbit/ Complete output (30 lines): running bdist_wheel running build running build_py creating build creating build/lib.macosx-10.9-x86_64-3.6 creating build/lib.macosx-10.9-x86_64-3.6/vowpalwabbit copying python/vowpalwabbit/pyvw.py -> build/lib.macosx-10.9-x86_64-3.6/vowpalwabbit copying python/vowpalwabbit/__init__.py -> build/lib.macosx-10.9-x86_64-3.6/vowpalwabbit copying python/vowpalwabbit/sklearn_vw.py -> build/lib.macosx-10.9-x86_64-3.6/vowpalwabbit running egg_info writing python/vowpalwabbit.egg-info/PKG-INFO writing dependency_links to python/vowpalwabbit.egg-info/dependency_links.txt writing top-level names to python/vowpalwabbit.egg-info/top_level.txt reading manifest file 'python/vowpalwabbit.egg-info/SOURCES.txt' reading manifest template 'MANIFEST.in' warning: no files found matching 'python/*.dll' under directory 'python/bin' warning: no files found matching 'python/packages.config' under directory 'python/windows*' warning: no files found matching 'python/*.vcxproj*' under directory 'python/windows*' warning: no previously-included files matching '*.o' found anywhere in distribution warning: no previously-included files matching '*.exe' found anywhere in distribution warning: no previously-included files matching '*.pyc' found anywhere in distribution warning: no previously-included files matching '*' found under directory 'vowpalwabbit/.nuget' warning: no previously-included files matching '*' found under directory 'vowpalwabbit/slim' writing manifest file 'python/vowpalwabbit.egg-info/SOURCES.txt' copying python/vowpalwabbit/conda_install.sh -> build/lib.macosx-10.9-x86_64-3.6/vowpalwabbit running build_ext creating build/temp.macosx-10.9-x86_64-3.6 cmake -DCMAKE_BUILD_TYPE=Release -DPY_VERSION=3.6 -DBUILD_PYTHON=On -DBUILD_TESTS=Off -DWARNINGS=Off -DBoost_NO_BOOST_CMAKE=ON -DBOOST_ROOT=/Users/someusername/anaconda3/envs/PYTH3610 -DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/private/var/folders/rw/g2c0r4fs0t73v457sqp_h2rh0000gn/T/pip-install-iuanczrt/vowpalwabbit/build/lib.macosx-10.9-x86_64-3.6 /private/var/folders/rw/g2c0r4fs0t73v457sqp_h2rh0000gn/T/pip-install-iuanczrt/vowpalwabbit unable to execute 'cmake': No such file or directory error: command 'cmake' failed with exit status 1

---------------------------------------- ERROR: Failed building wheel for vowpalwabbit Running setup.py clean for vowpalwabbit Failed to build vowpalwabbit Installing collected packages: vowpalwabbit Running setup.py install for vowpalwabbit ... error ERROR: Command errored out with exit status 1: command: /Users/someusername/anaconda3/envs/PYTH3610/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/rw/g2c0r4fs0t73v457sqp_h2rh0000gn/T/pip-install-iuanczrt/vowpalwabbit/setup.py'"'"'; __file__='"'"'/private/var/folders/rw/g2c0r4fs0t73v457sqp_h2rh0000gn/T/pip-install-iuanczrt/vowpalwabbit/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/rw/g2c0r4fs0t73v457sqp_h2rh0000gn/T/pip-record-9_5g1qrn/install-record.txt --single-version-externally-managed --compile cwd: /private/var/folders/rw/g2c0r4fs0t73v457sqp_h2rh0000gn/T/pip-install-iuanczrt/vowpalwabbit/ Complete output (30 lines): running install running build running build_py creating build creating build/lib.macosx-10.9-x86_64-3.6 creating build/lib.macosx-10.9-x86_64-3.6/vowpalwabbit copying python/vowpalwabbit/pyvw.py -> build/lib.macosx-10.9-x86_64-3.6/vowpalwabbit copying python/vowpalwabbit/__init__.py -> build/lib.macosx-10.9-x86_64-3.6/vowpalwabbit copying python/vowpalwabbit/sklearn_vw.py -> build/lib.macosx-10.9-x86_64-3.6/vowpalwabbit running egg_info writing python/vowpalwabbit.egg-info/PKG-INFO writing dependency_links to python/vowpalwabbit.egg-info/dependency_links.txt writing top-level names to python/vowpalwabbit.egg-info/top_level.txt reading manifest file 'python/vowpalwabbit.egg-info/SOURCES.txt' reading manifest template 'MANIFEST.in' warning: no files found matching 'python/*.dll' under directory 'python/bin' warning: no files found matching 'python/packages.config' under directory 'python/windows*' warning: no files found matching 'python/*.vcxproj*' under directory 'python/windows*' warning: no previously-included files matching '*.o' found anywhere in distribution warning: no previously-included files matching '*.exe' found anywhere in distribution warning: no previously-included files matching '*.pyc' found anywhere in distribution warning: no previously-included files matching '*' found under directory 'vowpalwabbit/.nuget' warning: no previously-included files matching '*' found under directory 'vowpalwabbit/slim' writing manifest file 'python/vowpalwabbit.egg-info/SOURCES.txt' copying python/vowpalwabbit/conda_install.sh -> build/lib.macosx-10.9-x86_64-3.6/vowpalwabbit running build_ext creating build/temp.macosx-10.9-x86_64-3.6 cmake -DCMAKE_BUILD_TYPE=Release -DPY_VERSION=3.6 -DBUILD_PYTHON=On -DBUILD_TESTS=Off -DWARNINGS=Off -DBoost_NO_BOOST_CMAKE=ON -DBOOST_ROOT=/Users/someusername/anaconda3/envs/PYTH3610 -DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/private/var/folders/rw/g2c0r4fs0t73v457sqp_h2rh0000gn/T/pip-install-iuanczrt/vowpalwabbit/build/lib.macosx-10.9-x86_64-3.6 /private/var/folders/rw/g2c0r4fs0t73v457sqp_h2rh0000gn/T/pip-install-iuanczrt/vowpalwabbit unable to execute 'cmake': No such file or directory error: command 'cmake' failed with exit status 1 ---------------------------------------- ERROR: Command errored out with exit status 1: /Users/someusername/anaconda3/envs/PYTH3610/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/rw/g2c0r4fs0t73v457sqp_h2rh0000gn/T/pip-install-iuanczrt/vowpalwabbit/setup.py'"'"'; __file__='"'"'/private/var/folders/rw/g2c0r4fs0t73v457sqp_h2rh0000gn/T/pip-install-iuanczrt/vowpalwabbit/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/rw/g2c0r4fs0t73v457sqp_h2rh0000gn/T/pip-record-9_5g1qrn/install-record.txt --single-version-externally-managed --compile Check the logs for full command output.

[peterychang]

@elviraux Unfortunately this is not an issue we can resolve from our end. Conda needs to rebuild python on a newer version of OSX, linking against libc++ (instead of libstdc++)

The only workaround at the moment is to avoid using anaconda on MacOS 10.14+

[peterychang]

One planned task may actually resolve this issue (this is likely how other libraries work despite the missing library).

Only the c++ library is incompatible, the underlying c library should be compatible. If we created a VW shared object with a pure C interface, and called those functions directly from the python wrappers, we should be able to resolve the crashes.

NOTE This is a long-term goal thats currently being discussed. There is no timeline or estimate for this change, but we will take this issue into account as we work on it.

[bmdy]

@peterychang Actually I ended up just changing my version of python and it is working fine now. Thanks for your help. But just for future reference, if there are instructions on building and using older version of boost against pyvw would appreciate it.

@scgalois Dear Scgalois, how do you change your version of python? I encounter the same problem, but cannot fix by now. Thank you very much for sharing your experience.

[bmdy]

There is a known issue on OSX 10.14+ with the currently deployed version of python on Anaconda.

The problem is as follows:

• Python requires linking binaries (eg: VW) to also link against the same c++ library it did.
• Anaconda builds python against an older version of OSX and links against libstdc++
• libstdc++ was deprecated by Apple/Clang several years ago. And as of OSX 10.14, the library was removed entirely.

As a result, VW will typically fail to compile or install on the latest version of OSX. Using compiler options to force the use of libc++ will cause python to crash.

At the moment, the only resolutions available to users are to either uninstall Anaconda and use Homebrew or to copy the missing libraries from an older version of xcode.

We are currently exploring ways to resolve these compatibility issues, but there does not appear to be a quick or easy solution to the problem.

If this is a critical issue for you, please feel free to reach out to us so we can see if there is an individual solution that may work.

We also welcome anybody who would like to contribute to the resolution of this issue.

Dear peterychang, I encounter the same problem and cannot fix by now. My MacOS is Mojave 10.14.6 with Python 3.7.7 and Clang 11.0.0. Can you help me try to resolve this issue? Thank you very much!

[peterychang]

@bmdy The only solution I know of is to uninstall Anaconda and use a compatible version of python, Brew's version should will work.

If that's not an option, you may be able to find a conda repo that has a version of python built against a newer version of OSX, but it seems doubtful since it would introduce incompatibilities with base Anaconda packages.

[bmdy]

@bmdy The only solution I know of is to uninstall Anaconda and use a compatible version of python, Brew's version should will work.

If that's not an option, you may be able to find a conda repo that has a version of python built against a newer version of OSX, but it seems doubtful since it would introduce incompatibilities with base Anaconda packages.

Thank you very much for your response. I've uninstalled Anaconda and use the Brew's version of Python. But I still encounter the problem: Library not loaded: @rpath/libpython3.7m.dylib. I've tried several ways to resolve this, but in vain. Can you help me in more details?

[peterychang]

@bmdy Have you rebuilt VW after uninstalling Anaconda? You'll need to pip uninstall vowpalwabbit before attempting to install it again.

If that doesn't work, please paste any output you get from the install process as well as the output of which python (or which python3, i'm not sure what the default for MacOS is)

[bmdy]

@bmdy Have you rebuilt VW after uninstalling Anaconda? You'll need to pip uninstall vowpalwabbit before attempting to install it again.

If that doesn't work, please paste any output you get from the install process as well as the output of which python (or which python3, i'm not sure what the default for MacOS is)

Yeah. I've completely uninstall vowpalwabbit. My process is as follow:

$ pip install vowpalwabbit

$ brew install cmake
$ brew install boost
$ brew install boost-python3

The default 'which python' gives '/usr/bin/python' and 'which python3' gives '/usr/local/bin/python3'.

Below is the output:
Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python3.7/site-packages/vowpalwabbit/pyvw.py", line 5, in
import pylibvw
ImportError: dlopen(/usr/local/lib/python3.7/site-packages/pylibvw.so, 2): Library not loaded: @rpath/libpython3.7m.dylib
Referenced from: /usr/local/lib/python3.7/site-packages/pylibvw.so
Reason: image not found

[peterychang]

I'm wondering if you got the cached version of the binary. Try running the following command (it'll dump the logs into a file so you can attach it here if needed)

pip -v install --force-reinstall --ignore-installed --no-binary :all: vowpalwabbit > vw_install.log

This will force pip to rebuild the VW binary. If it continues to crash, please attach the logs here copy and paste the output. Sorry, it looks like the install logs are going to stderr instead of stdout. You can just copy/paste the logs and I'll take a look

[peterychang]

For documentation's sake, the command
pip -v install --force-reinstall --ignore-installed --no-binary :all: vowpalwabbit 2> vw_install.log
will force a rebuild/reinstall and output the build logs to a log file.

[bmdy]

For documentation's sake, the command
pip -v install --force-reinstall --ignore-installed --no-binary :all: vowpalwabbit 2> vw_install.log
will force a rebuild/reinstall and output the build logs to a log file.

Thank you very much! It works. I can use vw to write code now. I will come to test more codes tomorrow and come back to you if some new issues occur.

[bmdy]

For documentation's sake, the command
pip -v install --force-reinstall --ignore-installed --no-binary :all: vowpalwabbit 2> vw_install.log
will force a rebuild/reinstall and output the build logs to a log file.

I've encountered a new problem. Now I need to write code on a server with Linux version 4.9.93. The python is Python 3.6.8 (with [GCC 7.3.0] on linux) with Miniconda3. How can I install vowpalwabbit on this environment? I've tried several ways following the instructions on this page https://pypi.org/project/vowpalwabbit/, but all failed. Thank you very much!

Maybe I should open a new problem. I opened the #2370. You can paste your answer there.

[jackgerrits]

I am currently investigating if it is feasible to get wheels for MacOS and have these preliminary artifacts - https://github.com/jackgerrits/vowpal_wabbit/actions/runs/254408712 if you could try installing the relevant wheel from there and see if it resolves the issue it would be appreciated

[jackgerrits]

We have wheels produced for each commit now. Once we release 8.9 they will make it to pip so you can simply pip install vowpalwabbit. In the meantime check the wiki for a link to the wheel artifacts: https://github.com/VowpalWabbit/vowpal_wabbit/wiki/Python#bleeding-edge-latest-commit-on-master

[jackgerrits]

Now that VW 8.9 is released, there are now wheels available. Using pip install vowpalwabbit should pick up these wheels and mitigate the conda issue.

If anyone hits this in future please feel free to comment/reopen the issue.