Pip inconsistent on whether it looks up a pre-release based on the order of specifiers

[notatallshaw]

Description

Pip will choose a pre-release for a package if the specifier of the package includes a pre-release version.

For example pandas>=2.1.5rc01 will currently install pandas-2.2.0rc0. But if there are two specifiers and one of them includes a pre-release and the other doesn't then the behavior is order dependent.

Expected behavior

If either version of the specifier involves a pre-release version the pre-release version should be installed

pip version

23.3.2

Python version

3.11

OS

Linux

How to Reproduce

1. Be on a date before Pandas 2.2.0 is installed or use pypi-timemachine to set the date to 2024-01-09
2. pip install --dry-run --no-deps --ignore-install "pandas>2.1.4" "pandas>=2.1.5rc0"
3. pip install --dry-run --no-deps --ignore-install "pandas>=2.1.5rc01" "pandas>2.1.4"

Output

pandas>2.1.4 pandas>=2.1.5rc0

pip install --dry-run --no-deps --ignore-install "pandas>2.1.4" "pandas>=2.1.5rc0"
ERROR: Could not find a version that satisfies the requirement pandas>2.1.4 (from versions: 0.1, 0.2, 0.3.0, 0.4.0, 0.4.1, 0.4.2, 0.4.3, 0.5.0, 0.6.0, 0.6.1, 0.7.0, 0.7.1, 0.7.2, 0.7.3, 0.8.0, 0.8.1, 0.9.0, 0.9.1, 0.10.0, 0.10.1, 0.11.0, 0.12.0, 0.13.0, 0.13.1, 0.14.0, 0.14.1, 0.15.0, 0.15.1, 0.15.2, 0.16.0, 0.16.1, 0.16.2, 0.17.0, 0.17.1, 0.18.0, 0.18.1, 0.19.0, 0.19.1, 0.19.2, 0.20.0, 0.20.1, 0.20.2, 0.20.3, 0.21.0, 0.21.1, 0.22.0, 0.23.0, 0.23.1, 0.23.2, 0.23.3, 0.23.4, 0.24.0, 0.24.1, 0.24.2, 0.25.0, 0.25.1, 0.25.2, 0.25.3, 1.0.0, 1.0.1, 1.0.2, 1.0.3, 1.0.4, 1.0.5, 1.1.0, 1.1.1, 1.1.2, 1.1.3, 1.1.4, 1.1.5, 1.2.0, 1.2.1, 1.2.2, 1.2.3, 1.2.4, 1.2.5, 1.3.0, 1.3.1, 1.3.2, 1.3.3, 1.3.4, 1.3.5, 1.4.0rc0, 1.4.0, 1.4.1, 1.4.2, 1.4.3, 1.4.4, 1.5.0rc0, 1.5.0, 1.5.1, 1.5.2, 1.5.3, 2.0.0rc0, 2.0.0rc1, 2.0.0, 2.0.1, 2.0.2, 2.0.3, 2.1.0rc0, 2.1.0, 2.1.1, 2.1.2, 2.1.3, 2.1.4, 2.2.0rc0)
ERROR: No matching distribution found for pandas>2.1.4

pandas>=2.1.5rc01 pandas>2.1.4

pip install --dry-run --no-deps --ignore-install "pandas>=2.1.5rc01" "pandas>2.1.4" 
Collecting pandas>=2.1.5rc01
  Using cached pandas-2.2.0rc0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (19 kB)
Using cached pandas-2.2.0rc0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (13.0 MB)
Would install pandas-2.2.0rc0

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[notatallshaw]

The problem here is probably compounded when Pip is backtracking through transitive dependencies, in that case the user has no control over what speicifers are being chosen and what order they are being compared to each other.

[pfmoore]

It feels like this and #12472 are basically two aspects of a more fundamental issue, which is that pre-release handling was never really "designed", but just grew somewhat organically¹.

I think this issue is big enough, and fundamental enough, that it probably needs someone to dedicate some significant time and energy to it, rather than trying to "fire-fight" individual issues. It would likely be a good candidate for being a funded project - and although it may be possible to do it on a volunteer basis, I suspect it would need someone who can dedicate "effectively full-time" energy to it.

My preference would be to structure work on changing pip's handling of pre-releases as a project. So start with a design document that reviews the existing standard, and looks at where that impacts pip's processing, and then write up what the desired behaviour for pip should be, pointing out where things don't currently conform to that. With that design as a roadmap, we can then look at how we implement it, addressing questions like backward compatibility, transition plans, handling of legacy and/or "oddball" projects², etc. Once all that planning and preparation has been done, then we can look at implementation.

That's a lot of work for an external contributor, but realistically I doubt the core pip maintainers will be able to commit to something like this in the foreseeable future. Having said that, though, I think we need to ensure that if someone does want to tackle this work, they have some pip maintainer willing to "sponsor" the work, in the sense of assisting with design questions, committing to reviews, and ensuring that PRs get merged in a timely manner. Otherwise, we risk someone doing a lot of work that then gets stalled because no-one is willing to merge it. And that's not a good outcome for anyone. I'd be willing to sponsor this work, but only if someone came up with a high level project plan that I felt was achievable, and could demonstrate that they had the project management skills to make it happen (specifically, I am not offering to be a project manager for this work).

Footnotes

1. In fact, I suspect that at least some of the logic pre-dates the version specifier standards. ↩

2. There used to be projects on PyPI that lived in a "permanent beta release" state, from what I recall. That may no longer be the case, but someone should check things like this and review whether the proposed behaviour is workable in the face of such projects. ↩

[notatallshaw]

I agree that for Pip to consistently fix pre-release handeling:

1. There should be a high level approach, with an end goal specified and agreed on
2. This would ideally be worked on as a single project
3. It is likely to be resource intensive (although who knows, maybe it's simpler than one expects)

However, I have some specific exceptions in mind which I think can, and should (if someone is willing), be done outside such a model:

• Document Pip's current behavior
• Add tests to Pip's current behavior
• Fix things which are more obviously bugs, even against Pip's current understanding of pre-releases as outlined by @pradyunsg, for example perhaps #12472 is a bug caused from either packaging or the way Pip is using packaging, maybe it will be fixed when Pip eventually vendors a new enough packaging release (and I have recently been testing and finding bugs in packaging, e.g. packaging#766 and packaging#768, I will investigate that example)

[pfmoore]

However, I have some specific exceptions in mind which I can be done outside such a model

-1 on documenting the current behaviour, much less classifying things as bugs because they don't behave like that documentation suggests they "should" do. That suggests you don't actually mean to document the current behaviour, but rather some idealised version of it which pip currently doesn't actually follow (but which still isn't what the spec says)!

[notatallshaw]

-1 on documenting the current behaviour, much less classifying things as bugs because they don't behave like that documentation suggests they "should" do. That suggests you don't actually mean to document the current behaviour, but rather some idealised version of it which pip currently doesn't actually follow (but which still isn't what the spec says)!

I don't understand this reply, as I understand it you seem to be interpreting what I wrote said multiple different ways, sorry for my confusion.

I am being very specific, when I say "Document Pip's current behavior, Add tests to Pip's current behavior". I mean have Pip actually docunent it's current behavior, with tests prooving that behavior. The goal of which is so that 1) Users don't misunderstand Pip's behavior from reading the packaging spec, and 2) Users don't have to figure out Pip's behavior from slow experiemental trial and error.

I do not mean Pip's behavior in it's entirity needs to be documented and exhuastively tested. But for example, the fact that Pip only considers a pre-release if not given a pre-release specifier or pre-release flag if there are only pre-releases for that package I don't think is documented? Any maybe not tested? And certainly not documented that it is a Pip specific implementation of the spec.

[pfmoore]

I do not mean Pip's behavior in it's entirity needs to be documented and exhuastively tested. But for example, the fact that Pip only considers a pre-release if not given a pre-release specifier or pre-release flag if there are only pre-releases for that package I don't think is documented? Any maybe not tested? And certainly not documented that it is a Pip specific implementation of the spec.

OK. I see what you're suggesting. But I don't want to document that, except in the form of a bug that states "pip doesn't follow the spec". I don't see the value in spending effort checking pip's conformance against a behaviour that's non-standard and which I'm not even sure is self-consistently defined.

There are genuine cases where it's not obvious what the correct behaviour should be. For example, when we have multiple specifiers, do we exclude pre-release versions from the list of considered versions? But that's just as much a problem with the spec as it is with pip's behaviour. We should be concentrating our efforts on addressing questions like that in the spec, not in pip.

We're going round in circles here. I'm going to try¹ to take a break from discussing this further for a few days. But my position is straightforward - I think there's a single pip issue, that we don't conform to the spec. Everything else should be tracked under that one issue², or considered a problem with the spec (to be solved there).

Footnotes

1. My track record for disengaging from discussions like this isn't good, but I'm going to do my best 🙂 ↩

2. And fixed by changing pip to conform to the spec. ↩

[notatallshaw]

I was going to just leave a Thumbs Up and not reply, but I don't think it provides clear enough context.

I don't have any further input and don't particularly object to your points.

[notatallshaw]

Sorry, I posted a couple of comments here that were factually incorrect, I have deleted them