Can't install on fedora 38: package versions have conflicting dependencies

[redmie]

I didn't manage to install LightZero on fedora 38 in an isolated environment (virtualenv).

This fails with the following error message:

INFO: pip is looking at multiple versions of lightzero to determine which version is compatible with other requirements. This could take a while.
ERROR: Cannot install di-engine[common-env]==0.4.9 and gym[all]==0.25.1 because these package versions have conflicting dependencies.

The conflict is caused by:
    di-engine[common-env] 0.4.9 depends on ale-py; extra == "common_env"
    gym[all] 0.25.1 depends on ale-py~=0.7.5; extra == "all"

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts

This can consistently be reproduced with this docker file:

FROM fedora:latest
RUN dnf install -y https://download1.rpmfusion.org/free/fedora/rpmfusion-free-release-$(rpm -E %fedora).noarch.rpm && dnf -y update && dnf -y install git python3-virtualenv python3-pip python3-devel ffmpeg-libs SDL2-devel SDL2_image-devel SDL2_mixer-devel SDL2_ttf-devel portmidi-devel libavdevice libavc1394-devel zlibrary-devel ccache mesa-libGL mesa-libGL-devel libjpeg-devel && dnf clean all
RUN uname -a
RUN git clone https://github.com/opendilab/LightZero
WORKDIR "/LightZero"
RUN python3 -m venv venv
RUN /LightZero/venv/bin/pip install -e .

Thanks for this amazing work! I plan to develop my own agents using lightzero as building block!

[puyuan1996]

Hello,

Thank you for your feedback. It appears that the primary cause of the version conflict in your Docker image might be due to the utilization of Python 3.11. Currently, Python 3.11 only supports versions 0.8.0 and 0.8.1 of ale-py, but not version 0.7.5. However, the LightZero library, which relies on the gym library (version 0.25.1), requires ale-py version 0.7.5, leading to a version mismatch.

To rectify this situation, I recommend you to switch the Python version in your Docker image to 3.8.x. This adjustment should help resolve the existing version conflict and ensure the proper functioning of the LightZero library and its dependencies.

Furthermore, I noticed that you are utilizing the Fedora operating system. While Fedora is a commendable OS, it's likely that LightZero has been primarily tested and optimized for Ubuntu and macOS. Therefore, the library may not have undergone comprehensive testing and verification on Fedora, which could result in unforeseen issues.

Considering these factors, unless there are special requirements or considerations, I suggest you consider switching to the Ubuntu operating system. Ubuntu has broader support in the open-source community, and most Python libraries and packages are well-supported and optimized for it.

I hope these suggestions are helpful!

Best regards!

[puyuan1996]

• A possible ubuntu+python3.8+lightzero docker file is something like the following:

# Start from Ubuntu 20.04
FROM ubuntu:20.04

# Set the working directory
WORKDIR /app

# Install Python 3.8 and other dependencies
RUN apt-get update && \
    apt-get install -y python3.8 python3-pip gcc g++ swig git && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*

# Create a symlink for Python
RUN ln -s /usr/bin/python3.8 /usr/local/bin/python && \
    ln -s /usr/bin/pip3 /usr/local/bin/pip

# Update pip and setuptools
RUN python -m pip install --upgrade pip setuptools

# Clone and install the package in editable mode
RUN git clone https://github.com/opendilab/LightZero.git
RUN pip install -e ./LightZero

# The command to run when this image starts up
CMD ["python", "./LightZero/zoo/classic_control/cartpole/config/cartpole_muzero_config.py"]

# docker build cmd:
# docker build -t ubuntu-py38-lz .

[puyuan1996]

Thank you for raising this issue. We have addressed this in a pull request (PR). In this PR, we have added an example Dockerfile along with detailed usage instructions. This will help you set up an environment with all the dependencies needed to run the LightZero library. Please refer to this PR for more information.

We appreciate your feedback, it's incredibly valuable to our project. If you have any more questions or need further assistance, please don't hesitate to let us know.

[redmie]

Thanks you.
It was indeed a python version problem. It works with python 3.8!

To follow this up, I did some tests using python 3.7 to python 3.11.

Install fails with python 3.7 with the following error:

Obtaining file:///home/rpiau/Documents/reinforcement_learning/LightZero
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... done
  Preparing editable metadata (pyproject.toml) ... done
Collecting DI-engine[common_env]>=0.4.9
  Using cached DI_engine-0.4.9-py3-none-any.whl (2.1 MB)
Collecting pympler
  Using cached Pympler-1.0.1-py3-none-any.whl (164 kB)
ERROR: Ignored the following versions that require a different python version: 1.22.0 Requires-Python >=3.8; 1.22.1 Requires-Python >=3.8; 1.22.2 Requires-Python >=3.8; 1.22.3 Requires-Python >=3.8; 1.22.4 Requires-Python >=3.8; 1.23.0 Requires-Python >=3.8; 1.23.0rc1 Requires-Python >=3.8; 1.23.0rc2 Requires-Python >=3.8; 1.23.0rc3 Requires-Python >=3.8; 1.23.1 Requires-Python >=3.8; 1.23.2 Requires-Python >=3.8; 1.23.3 Requires-Python >=3.8; 1.23.4 Requires-Python >=3.8; 1.23.5 Requires-Python >=3.8; 1.24.0 Requires-Python >=3.8; 1.24.0rc1 Requires-Python >=3.8; 1.24.0rc2 Requires-Python >=3.8; 1.24.1 Requires-Python >=3.8; 1.24.2 Requires-Python >=3.8; 1.24.3 Requires-Python >=3.8; 1.24.4 Requires-Python >=3.8; 1.25.0 Requires-Python >=3.9; 1.25.0rc1 Requires-Python >=3.9; 1.25.1 Requires-Python >=3.9; 1.25.2 Requires-Python >=3.9; 1.26.0b1 Requires-Python <3.13,>=3.9; 1.26.0rc1 Requires-Python <3.13,>=3.9
ERROR: Could not find a version that satisfies the requirement numpy>=1.24.4 (from lightzero) (from versions: 1.3.0, 1.4.1, 1.5.0, 1.5.1, 1.6.0, 1.6.1, 1.6.2, 1.7.0, 1.7.1, 1.7.2, 1.8.0, 1.8.1, 1.8.2, 1.9.0, 1.9.1, 1.9.2, 1.9.3, 1.10.0.post2, 1.10.1, 1.10.2, 1.10.4, 1.11.0, 1.11.1, 1.11.2, 1.11.3, 1.12.0, 1.12.1, 1.13.0, 1.13.1, 1.13.3, 1.14.0, 1.14.1, 1.14.2, 1.14.3, 1.14.4, 1.14.5, 1.14.6, 1.15.0, 1.15.1, 1.15.2, 1.15.3, 1.15.4, 1.16.0, 1.16.1, 1.16.2, 1.16.3, 1.16.4, 1.16.5, 1.16.6, 1.17.0, 1.17.1, 1.17.2, 1.17.3, 1.17.4, 1.17.5, 1.18.0, 1.18.1, 1.18.2, 1.18.3, 1.18.4, 1.18.5, 1.19.0, 1.19.1, 1.19.2, 1.19.3, 1.19.4, 1.19.5, 1.20.0, 1.20.1, 1.20.2, 1.20.3, 1.21.0, 1.21.1, 1.21.2, 1.21.3, 1.21.4, 1.21.5, 1.21.6)
ERROR: No matching distribution found for numpy>=1.24.4

But works with python 3.8 and 3.9 and also surprisingly with 3.10.
And fails as predicted with python 3.11.

I have the same test results for every version 3 errors, 1 warning.

I propose to change version bounds to >=3.8, <3.11 as this seems to work.
Please check the related PR

[puyuan1996]

Hello, we sincerely appreciate your feedback and suggestion. We will carefully review this pull request and make the necessary updates to the Python version requirement after conducting thorough testing. We also welcome any additional suggestions or improvements you may have. Best wishes to you.