[python-package] Installation instructions and process critically flawed

[nickion]

Description

Trying to build a GPU aware version with python library is hampered by faulty documentation and procedures.

1. On pypi.org the instruction pip install lightgbm --install-option=--gpu is incorrect as --install-option was removed some time ago.
2. Trying to build the pythion component with python setup.py install fails with

python setup.py install
Traceback (most recent call last):
File "/home/dev/LightGBM/python-package/setup.py", line 300, in
version = _version_file.read_text(encoding='utf-8').strip()
File "/home/dev/miniconda3/envs/tf/lib/python3.10/pathlib.py", line 1134, in read_text
with self.open(mode='r', encoding=encoding, errors=errors) as f:
File "/home/dev/miniconda3/envs/tf/lib/python3.10/pathlib.py", line 1119, in open
return self._accessor.open(self, mode, buffering, encoding, errors,
FileNotFoundError: [Errno 2] No such file or directory: '/home/dev/LightGBM/python-package/lightgbm/VERSION.txt'

Reproducible example

1. Visit the pypi.org lightgbm page (possibly elsewhere too), reading and trying to follow the instructions for Linux with a github package
2. cd python-package ; python setup.py install

Environment info

LightGBM version or commit hash:

3.3.5.99

The repo was cloned and I went from there.
I have successfully built the library with cmake, but how to build / install a python library is unclear given the errors above.

[nickion]

To install I replaced the default shared library with the one built, and that was successful for the python module to function, albeit with other issues.

[jameslamb]

Thanks for using LightGBM and sorry for the confusion.

The documentation at https://pypi.org/project/lightgbm/ corresponds to the last released version of LightGBM, in this case v3.3.5. It's drifted significantly from the source code here on GitHub because it has been a very long time since the last substantive release of LightGBM. We're trying to work towards a next release (v4.0.0). You can subscribe to #5153 to be notified when that happens.

--install-option was removed some time ago.

pip install --install-option was still supported as of when lightgbm v3.3.5 was released (January 2023)... that support was only dropped by pip v23.1 (April 2023, see https://pip.pypa.io/en/stable/news/#v23-1).

To build a GPU-enabled version of lightgbm from the source distribution on PyPI, using pip, you'll have to downgrade to an older version pip, e.g.

pip install 'pip<23.1'
pip install \
    --no-binary lightgbm \
    --install-option=--gpu \
    lightgbm

Or you can build lib_lightgbm.so from source first and then install a version of the Python package using it. Running the following from the root of the repo:

git clone --recursive https://github.com/microsoft/LightGBM.git
mkdir ./LightGBM/build
cd ./LightGBM/build
cmake -DUSE_GPU=ON ..
make -j2 _lightgbm
cd ..
sh ./build-python.sh install --precompile

Trying to build the python component with python setup.py install fails

Support for directly running python setup.py {command} in LightGBM sources was removed in #5837 . You can read #5061 for more details, but essentially setuptools is removing support for using a setup.py file as a command-line interface, so in this project we're in the process of doing the same.

To install from sources cloned from GitHub, follow the docs here in GitHub: https://github.com/microsoft/LightGBM/tree/master/python-package.

albeit with other issues

If you're able to provide a reproducible example or any details explaining what "other issues" means, we'd be happy to try to help.

[github-actions]

This issue has been automatically closed because it has been awaiting a response for too long. When you have time to to work with the maintainers to resolve this issue, please post a new comment and it will be re-opened. If the issue has been locked for editing by the time you return to it, please open a new issue and reference this one. Thank you for taking the time to improve LightGBM!

[github-actions]

This issue has been automatically locked since there has not been any recent activity since it was closed. To start a new related discussion, open a new issue at https://github.com/microsoft/LightGBM/issues including a reference to this.