Requires-Python mis-evaluated

[dimbleby]

Description

A package with Requires-Python: >=3.7,<=3.10 cannot be installed at python 3.10.4:

ERROR: Package 'til-bigquery' requires a different Python: 3.10.4 not in '<=3.10,>=3.7'

Expected behavior

My reading of PEP-345 is that this should be fine:

When a version is provided, it always includes all versions that starts with the same value

So <=3.10 should be equivalent to <3.11.0, not to <=3.10.0

pip version

22.2.2

Python version

3.10.4

OS

Ubuntu

How to Reproduce

$ curl --user __token__:glpat-mkEPJ4Rsy2peTCrH23pG --remote-name "https://gitlab.com/api/v4/projects/38869805/packages/pypi/files/7a4731d831d4b37262481002271e359f96017570e9480ef16c89489e0b41252f/til_bigquery-0.3.4-py3-none-any.whl"

$ pip install til_bigquery-0.3.4-py3-none-any.whl

Output

$ curl --user __token__:glpat-mkEPJ4Rsy2peTCrH23pG --remote-name "https://gitlab.com/api/v4/projects/38869805/packages/pypi/files/7a4731d831d4b37262481002271e359f96017570e9480ef16c89489e0b41252f/til_bigquery-0.3.4-py3-none-any.whl"
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  4976  100  4976    0     0   9808      0 --:--:-- --:--:-- --:--:--  9795

$ pip install til_bigquery-0.3.4-py3-none-any.whl
Processing ./til_bigquery-0.3.4-py3-none-any.whl
Collecting structlog>=20.2.0
  Using cached structlog-22.1.0-py3-none-any.whl (58 kB)
Collecting backoff>=1.11.1
  Using cached backoff-2.1.2-py3-none-any.whl (14 kB)
Collecting google-cloud-bigquery<3.0.0,>=2.9.0
  Using cached google_cloud_bigquery-2.34.4-py2.py3-none-any.whl (206 kB)
Collecting pydantic>=1.7.3
  Using cached pydantic-1.9.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (12.0 MB)
ERROR: Package 'til-bigquery' requires a different Python: 3.10.4 not in '<=3.10,>=3.7'

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[sbidoul]

PEP 440 talks of zero-padding, so the current behavior looks correct to me.

[uranusjr]

I think the line you quoted applies only specifically to this form:

Requires-Python: 3.10

i.e. no operator, just one single version. If an operator is found in the value, PEP 440 zero-padding applies. I guess you could argue that it’s slightly confusing if looked from a certain angle (Python 3.10.4 is a Python 3.10 so <=3.10 can be interpreted to include it), but that’s not how PEP 440 defines things, so the current behaviour is correct.

[dimbleby]

it's surprising that <=x would exclude things that are included by x - are you sure that's what's intended? The former looks as though it should be strictly larger.

I don't see where PEP440 says that zero-padding applies in only one of those circumstances - indeed the linked section explicitly says (emphasis is mine)

_As with version matching_, the release segment is zero padded

So if PEP440 applies here then isn't 3.10 going to have to exclude 3.10.4?

Edit, one more thing, re

I think the line you quoted applies only specifically to this form:

I think it doesn't, I think it very clearly uses the word "always"!

[dimbleby]

did a little more digging, I think pip's behaviour is that 3.10 allows 3.10.any, whereas ==3.10 allows only 3.10.0.

DId I understand correctly? Is this behaviour justifiable?

[dstufft]

The release padding means adding extra zeros to make things equal to the same length, since 3.10 has less segments then 3.10.4, it needs padded out to 3.10.0, which means that <=3.10 is equivalent to <=3.10.0.

When you write that out, it makes more sense I think, since you're effectively evaluating 3.10.4<=3.10.0.

[dimbleby]

I see that's what PEP440 says, but I think it's in disagreement with PEP345.

'Later wins' is fair enough: but then how to have 3.10 mean 3.10.any?

Perhaps the idea is that PEP440 doesn't have anything to say about the naked form 3.10, so this is the sole case where PEP345's "always includes all versions that starts with the same value" lives on?

[RonnyPfannschmidt]

Given the error potential / ease of misunderstanding, should there be a mandate for warning of intuitive misuse of the zero padding so people may easily get accidental min pins

[dstufft]

To be honest, PEP 345 is from an era when the specs were less rigorously defined ¹, and thus the specification in PEP 345 leaves a lot of edge cases or even normal cases somewhat ill defined. However, PEP 440 came after PEP 345 and it supersedes PEP 345 and explicitly doesn't define a "bare" comparison operator, which means two things:

1. Where possible, packaging tools should be using PEP 440 semantics and should ignore PEP 345 semantics.
2. PEP 440 does not define a "bare" comparison, instead choosing to mandate that all specifiers have a comparison operator. What that means for packages using a "bare" comparisons is not defined, presumably tools would either error on them or attempt to convert them to PEP 440 form (hopefully losslessly, but it might be a lossy conversion).

Given that the Requires-Python in question does not use a bare comparison, (2) is not relevant, which means that it falls under (1), and PEP 345 version comparisons no longer matter, since PEP 440 supersedes. Which means that when comparing against a version like 3.10.4, we need to do zero padding to turn >=3.7,<=3.10 into >=3.7.0,<=3.10.0, in which case I believe it becomes very obvious that 3.10.4 should not match that specifier.

Footnotes

1. Not that we do a perfect job of this today, but we're better than we were at producing more precise specifications. ↩

[dimbleby]

thanks for the discussion