-
Notifications
You must be signed in to change notification settings - Fork 2.9k
Document why uv discards upper bounds on requires-python
#15927
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -156,26 +156,62 @@ Windows, Linux and macOS. | |
| To ensure that a resolution with `requires-python = ">=3.9"` can actually be installed for the | ||
| included Python versions, uv requires that all dependencies have the same minimum Python version. | ||
| Package versions that declare a higher minimum Python version, e.g., `requires-python = ">=3.10"`, | ||
| are rejected, because a resolution with that version can't be installed on Python 3.9. For | ||
| simplicity and forward compatibility, only lower bounds in `requires-python` are respected. For | ||
| example, if a package declares `requires-python = ">=3.8,<4"`, the `<4` marker is not propagated to | ||
| the entire resolution. | ||
|
|
||
| This default is a problem for packages that use the version-dependent C API of CPython, such as | ||
| numpy. Each numpy release support 4 Python minor versions, e.g., numpy 2.0.0 has wheels for CPython | ||
| 3.9 through 3.12 and declares `requires-python = ">=3.9"`, while numpy 2.1.0 has wheels for CPython | ||
| 3.10 through 3.13 and declares `requires-python = ">=3.10"`. The means that when we resolve a | ||
| `numpy>=2,<3` requirement in a project with `requires-python = ">=3.9"`, we resolve numpy 2.0.0 and | ||
| the lockfile doesn't install on Python 3.13 or newer. To alleviate this, whenever we reject a | ||
| version due to a too high Python requirement, we fork on that Python version. This behavior is | ||
| controlled by `--fork-strategy`. In the example case, upon encountering numpy 2.1.0 we fork into | ||
| Python versions `>=3.9,<3.10` and `>=3.10` and resolve two different numpy versions: | ||
| are rejected, because a resolution with that version can't be installed on Python 3.9. This ensures | ||
| that when you are on an old Python version, you can install old packages, instead of getting newer | ||
| packages that require newer Python syntax or standard library features. | ||
|
|
||
| uv ignores upper-bounds on `requires-python`, with special handling for packages with only | ||
| ABI-specific wheels. For example, if a package declares `requires-python = ">=3.8,<4"`, the `<4` | ||
| part is ignored. There is a detailed discussion with drawbacks and alternatives in | ||
| [#4022](https://github.com/astral-sh/uv/issues/4022) and this | ||
| [DPO thread](https://discuss.python.org/t/requires-python-upper-limits/12663), this section | ||
| summarizes the aspects most relevant to uv's design. | ||
|
|
||
| For most projects, it's not possible to determine whether they will be compatible with a new version | ||
| before it's released, so blocking newer versions in advance would block users from upgrading or | ||
| testing newer Python versions. The exceptions are packages which use the unstable C ABI or internals | ||
| of CPython such as its bytecode format. | ||
|
|
||
| Introducing a `requires-python` upper bound to a project that previously wasn't using one will not | ||
| prevent the project from being used on a too recent Python version. Instead of failing, the resolver | ||
| will pick an older version without the bound, circumventing the bound. | ||
|
|
||
| For the resolution to be as universally installable as possible, uv ensures that the selected | ||
| dependency versions are compatible with the `requires-python` range of the project. For example, for | ||
| a project with `requires-python = ">=3.12"`, uv will not use a dependency version with | ||
| `requires-python = ">=3.13"`, as otherwise the resolution is not installable on Python 3.12, which | ||
| the project declares to support. Applying the same logic to upper bounds means that bumping the | ||
| upper Python version bound on a project makes it compatible with less dependency versions, | ||
| potentially failing to resolve when no version of a dependency supports the required range. (Bumping | ||
| the lower Python version bound has the inverse effect, it only increases the set of supported | ||
| dependency versions.) | ||
|
|
||
| Note that this is different for Conda, as the Conda solver also determines the Python version, so it | ||
| can choose a lower Python version instead. Conda can also change metadata after a release, so it can | ||
| update compatibility for a new Python version, while metadata on PyPI cannot be changed once | ||
| published. | ||
|
|
||
| Ignoring an upper bound is a problem for packages such as numpy which use the version-dependent C | ||
| API of CPython. As of writing, each numpy release support 4 Python minor versions, e.g., numpy 2.0.0 | ||
| has wheels for CPython 3.9 through 3.12 and declares `requires-python = ">=3.9"`, while numpy 2.1.0 | ||
| has wheels for CPython 3.10 through 3.13 and declares `requires-python = ">=3.10"`. The means that | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Thanks! |
||
| when uv resolves a `numpy>=2,<3` requirement in a project with `requires-python = ">=3.9"`, it | ||
| selects numpy 2.0.0 and the lockfile doesn't install on Python 3.13 or newer. To alleviate this, | ||
| whenever uv rejects a version that requires a newer Python version, we fork by splitting the | ||
| resolution markers on that Python version. This behavior can be controlled by `--fork-strategy`. In | ||
| the example case, upon encountering numpy 2.1.0 we fork into Python versions `>=3.9,<3.10` and | ||
|
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The irony that we're using an upper bound on Python here is not lost on me, though for package selection we're only using the lower bound again, the upper bound exists for internal coherence tracking (and for a lockfile optimization where we drop wheels outside the range). |
||
| `>=3.10` and resolve two different numpy versions: | ||
|
|
||
| ``` | ||
| numpy==2.0.0; python_version >= "3.9" and python_version < "3.10" | ||
| numpy==2.1.0; python_version >= "3.10" | ||
| ``` | ||
|
|
||
| There's one case where uv does consider the upper bound: When the project uses an upper bound on | ||
| requires Python, such as `requires-python = "==3.13.*"` for an application that only deploys to | ||
| Python 3.13. uv prunes wheels from the lockfile that are outside the range (e.g., `cp312` and | ||
| `cp314`) in a post-processing step, which does not influence the resolution itself. | ||
|
|
||
| ## URL dependencies | ||
|
|
||
| In uv, a dependency can either be a registry dependency, a package with a version specifier or the | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm ignoring the option to split this into two fields here, or other advanced options such as ignoring the upper bound only on the current workspace, this is better documented in the DPO thread, and the other answer is that it's not worth doing that, if we wanted to add new metadata for e.g. the numba case then this should be a new PEP imho, so it's also rather for the DPO discussion.