[HELP WANTED] What's the recommended platform & python environment?

[mokeeqian]

Description

I have tried again and again on my local Windows 10 OS, using Conda python3.6 and venv python3.6, but still faild to run the project, due to the conflict version of Numpy and other problems.

So, are there anyone successfuily run it on Windows?
I need your help!

Other Comments

By the way, should I try the Linux platform, or any other way???

[gramhagen]

can you provide some info on what you're trying to run and any resulting error messages you see?
we would expect most examples to work under linux and windows (however running the spark based examples locally on a windows machine is usually more problematic)

[mokeeqian]

can you provide some info on what you're trying to run and any resulting error messages you see? we would expect most examples to work under linux and windows (however running the spark based examples locally on a windows machine is usually more problematic)

ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 88 from C header, got 80 from PyObject

occurs when I run the /examples/benchmark/moivelens.ipynb, of the first cell, It seems that, the Numpy's version has conflicted to other package, my current Numpy's version is 1.19.5
And than, I tried to upgrade and downgrade the Numpy, it does not work...

[gramhagen]

@laserprec we saw this in the tests a while back, I know there was a conda package that fixed this, but I think there was also another approach to resolve this numpy problem. Do you have any suggestions here?

[mokeeqian]

by the way, if I change the Numpy's version, instead of 1.19.5, it conflicts to the other packages that the recommenders requires, such as TF.
And, hopefully, I wish MicroSoft to detailly show the Install instructions via the SETUP.md to Windows or Linux local meachine users.
I would be gratiful for that!

[laserprec]

@mokeeqian I am sorry about the frustration you are experiencing. Yes, our documentation is scattered across different places and it is a much needed improvement to clean up the instructions.

By the way, should I try the Linux platform

I would recommend trying the installation on Linux in a virtual environment, and the installation instructions for Linux containers from Setup.md can also be used for any Linux machine:

sudo apt-get -y update
sudo apt-get -y install python3.6
sudo apt-get -y install python3-pip
sudo apt-get -y install python3.6-venv
sudo apt-get -y install libpython3.6-dev
sudo apt-get -y install cmake
sudo apt-get install -y libgomp1 openjdk-8-jre

python3.6 -m venv .env
source .env/bin/activate
pip install --upgrade pip setuptools wheel

export PYSPARK_DRIVER_PYTHON=$VIRTUAL_ENV/bin/python
export PYSPARK_PYTHON=$VIRTUAL_ENV/bin/python
echo "export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64" >> .env/bin/activate
echo "export PYSPARK_PYTHON=$VIRTUAL_ENV/bin/python" >> .env/bin/activate

pip install recommenders[all]

After running these commands, you should have an virtual environment .env set up for use in your current working directory. Assuming you have cloned our repo and it's your working directory, when you run the notebooks in our repo, just make sure you have activated this virtual environment (i.e. you ran source .env/bin/activate) and run:

jupyter notebook examples

I hope this should set up the notebook environment correctly. Please do let us know if you run into any issues!

[mokeeqian]

@mokeeqian I am sorry about the frustration you are experiencing. Yes, our documentation is scattered across different places and it is a much needed improvement to clean up the instructions.

By the way, should I try the Linux platform

I would recommend trying the installation on Linux in a virtual environment, and the installation instructions for Linux containers from Setup.md can also be used for any Linux machine:

sudo apt-get -y update
sudo apt-get -y install python3.6
sudo apt-get -y install python3-pip
sudo apt-get -y install python3.6-venv
sudo apt-get -y install libpython3.6-dev
sudo apt-get -y install cmake
sudo apt-get install -y libgomp1 openjdk-8-jre

python3.6 -m venv .env
source .env/bin/activate
pip install --upgrade pip setuptools wheel

export PYSPARK_DRIVER_PYTHON=$VIRTUAL_ENV/bin/python
export PYSPARK_PYTHON=$VIRTUAL_ENV/bin/python
echo "export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64" >> .env/bin/activate
echo "export PYSPARK_PYTHON=$VIRTUAL_ENV/bin/python" >> .env/bin/activate

pip install recommenders[all]

After running these commands, you should have an virtual environment .env set up for use in your current working directory. Assuming you have cloned our repo and it's your working directory, when you run the notebooks in our repo, just make sure you have activated this virtual environment (i.e. you ran source .env/bin/activate) and run:

jupyter notebook examples

I hope this should set up the notebook environment correctly. Please do let us know if you run into any issues!

Thank you very much!
I will try later just as you say, and give the next feedback.
Thanks again!

[laserprec]

@laserprec we saw this in the tests a while back, I know there was a conda package that fixed this, but I think there was also another approach to resolve this numpy problem. Do you have any suggestions here?

@gramhagen, is this the error you are referring to? If so, I think this is a different one that I haven't encountered before. A preliminary search on the internet suggests that it could be a pip problem. I would suggest following the instructions in the thread above to install via a virtual environment to try to eliminate any corrupted binary paths or build artifacts.

[gramhagen]

Is it possible to use numpy >= 1.20

[aptlin]

Is there any reason to limit python to < 3.8?

[mokeeqian]

Is it possible to use numpy >= 1.20

Could not work

[laserprec]

Is there any reason to limit python to < 3.8?

We currently have a dependency on TF v1.15, which does not support python 3.8+. We are working on an upgrade to TF2, so 3.8 will be in the route map after it's done.

[gramhagen]

latest release should address issues with numpy compatibility (as well as tf2 support), closing this issue now, but feel free to open another one if there are problems using the lastest version