Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[python-package] prevent build-python.sh from modifying already-installed dependencies #6403

Merged
merged 2 commits into from
Apr 20, 2024

Conversation

jameslamb
Copy link
Collaborator

The recommended workflow for building the Python package from source here uses the build-python.sh script, like this:

cmake -B build -S .
cmake --build build --target _lightgbm -j4
sh ./build-python.sh install --precompile

The version for the library from any two unreleased commits is the same (e.g. 4.3.0.99). To ensure that pip does not give up on successive runs and say "oh lightgbm==4.3.0.99 is already installed, I'm not installing this wheel", in #5907 we added --force-reinstall to the pip install dist/* in build-python.sh.

That has the unfortunate side-effect of also force-reinstalling all of lightgbm's strong runtime dependencies (numpy and scipy). That's undesirable for 2 reasons:

  • it's surprising to get your environment's numpy version upgraded by installing lightgbm, even when lightgbm is compatible with the version of numpy you already have
  • in a conda environment, this can result in a numpy conda package being replaced by a numpy wheel, which can cause a variety of linking and loading problems

This PR proposes avoiding that, by instead having build-python.sh install run pip uninstall --yes lightgbm.

How I tested this

In a conda environment without lightgbm installed, but with scipy and numpy installed, ran the following.

cmake -B build -S .
cmake --build build --target _lightgbm -j4
sh ./build-python.sh install --precompile

Saw this in the logs, confirming that it didn't change my numpy or scipy versions.

Requirement already satisfied: numpy in /Users/jlamb/miniforge3/envs/lgb-dev/lib/python3.11/site-packages (from lightgbm) (1.26.4)
Requirement already satisfied: scipy in /Users/jlamb/miniforge3/envs/lgb-dev/lib/python3.11/site-packages (from lightgbm) (1.12.0)

@jameslamb jameslamb merged commit f6ecd4d into master Apr 20, 2024
42 checks passed
@jameslamb jameslamb deleted the python-install-script branch April 20, 2024 02:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants