pip_parse builds wheels with different names

[linzhp]

🐞 bug report

Affected Rule

pip_parse

Is this a regression?

No

Description

When building PyPika 0.48.9, setuptools 75.8.0 creates a wheel called PyPika-0.48.9-py2.py3-none-any.whl, while 75.8.1 creates pypika-0.48.9-py2.py3-none-any.whl (pypa/setuptools#4766). When isolated = True, pip_parse relies on pip to determine which version of setuptools to build wheels of PyPika from source. At some point, pip decided to switch from 75.8.0 to 75.8.1. This caused some machines having PyPika-0.48.9-py2.py3-none-any.whl while some others having pypika-0.48.9-py2.py3-none-any.whl, leading to different checksums and cache keys during Bazel query/build.

Can we force pip to use the version of setuptools managed by Bazel?

🔬 Minimal Reproduction

By installing specific setuptools version and run pip wheel in non-isolated mode, we can see the different wheel names created:

(myenv) ➜  myenv pip install setuptools==75.8.0
...
Successfully installed setuptools-75.8.0
(myenv) ➜  myenv pip wheel --no-build-isolation pypika
...
Building wheels for collected packages: pypika
  Building wheel for pypika (PEP 517) ... done
  Created wheel for pypika: filename=PyPika-0.48.9-py2.py3-none-any.whl size=53771 sha256=976845cf855a2cff476a284ac81f0824ce3ba8bd69c5090d07fc302c0edcd042
  Stored in directory: /home/user/.cache/pip/wheels/3b/59/32/380c4d773968f655e18971a44e5a99f96dc2b9fb76287dae59
Successfully built pypika
(myenv) ➜  myenv rm -r /home/user/.cache/pip/wheels/3b/59/32/380c4d773968f655e18971a44e5a99f96dc2b9fb76287dae59 
(myenv) ➜  myenv rm PyPika-0.48.9-py2.py3-none-any.whl
(myenv) ➜  myenv pip install setuptools==75.8.1
...
Installing collected packages: setuptools
  Attempting uninstall: setuptools
    Found existing installation: setuptools 75.8.0
    Uninstalling setuptools-75.8.0:
      Successfully uninstalled setuptools-75.8.0
Successfully installed setuptools-75.8.1
(myenv) ➜  myenv pip wheel --no-build-isolation pypika                                                
...
Building wheels for collected packages: pypika
  Building wheel for pypika (PEP 517) ... done
  Created wheel for pypika: filename=pypika-0.48.9-py2.py3-none-any.whl size=53769 sha256=8f667e0f5b564a903d5116cf150cf80fd2dc938db10c4ec485d070c8fb2607c7
  Stored in directory: /home/user/.cache/pip/wheels/3b/59/32/380c4d773968f655e18971a44e5a99f96dc2b9fb76287dae59
Successfully built pypika

🌍 Your Environment

Operating System:

  
Linux
  

Output of bazel version:

  
7.3.1
  

Rules_python version:

  
0.39.0
  

Anything else relevant?

[aignas]

#2410 should include this, however no one is working on this yet.

I am inclined to close as duplicate, but will leave open if you want to submit fixes to the repo rules.

EDIT: Some extra thoughts on this:

• pip is vendoring all of the deps with itself, so what gets loaded should be deterministic?
• If it is not, I wonder if the PYTHON path that gets constructed in the pypi_repo_utils is non-deterministic.
• Maybe we should remove pip from hermetic toolchain, which get rid of those vendored extra deps. Maybe having a completely clean interpreter help with determinism here, but it is somewhat a spray and pray strategy...
• are there any blocking issues that would hold you from testing if the bug is reproducible with the latest rules_python version?

[linzhp]

pip is vendoring all of the deps with itself, so what gets loaded should be deterministic?

From the documentation, it's isolated but I don't think it's deterministic. Pip reads build-system.requires key in the pyproject.toml to see if it requires specific version of setuptools. In the example, it specifies "setuptools ~= 58.0". So pip will get the latest version the range of 58.0.x. In the case of PyPika, it doesn't specify setuptools version, so pip will get the latest version available in PyPI. The "latest" mean different version if we build at different times.

are there any blocking issues that would hold you from testing if the bug is reproducible with the latest rules_python version?

I just verify it in rules_python 1.2.0. I can see that pip still creates pypika-0.48.9-py2.py3-none-any.whl, which indicates that it's using setuptools 75.8.1 or later, while the setuptools that comes with rules_python is 70.0.0:

rules_python/python/private/pypi/deps.bzl

Line 79 in a816962

"https://files.pythonhosted.org/packages/de/88/70c5767a0e43eb4451c2200f07d042a4bcd7639276003a9c54a68cfcc1f8/setuptools-70.0.0-py3-none-any.whl",