Skip to content

Respect .python-version in pip-compile#14624

Closed
staticf0x wants to merge 3 commits intoastral-sh:mainfrom
staticf0x:pip-compile-respect-python-version
Closed

Respect .python-version in pip-compile#14624
staticf0x wants to merge 3 commits intoastral-sh:mainfrom
staticf0x:pip-compile-respect-python-version

Conversation

@staticf0x
Copy link

@staticf0x staticf0x commented Jul 15, 2025

Summary

This is an attempt to make uv pip compile respect .python-version file as reported in #9581.

The problem at hand is more complex than I anticipated, because you can specify multiple source files, which makes it difficult to "guess" where the .python-version might be. I chose this approach:

  1. Try just .python-version in cwd (for simple invocations like uv pip compile requirements.in)
  2. Try to find a sibling to the first provided source file

I am more than open to suggestions how to make this better, though, it doesn't feel exactly right.

Test Plan

Given this requirements.in:

requests
rich ; python_version >= '3.12'

and .python-version:

3.11

My machine has Python 3.9, 3.11, 3.12 and 3.13 (default) installed. When I run uv -v pip compile requirements.in before this PR, I get:

DEBUG Starting Python discovery for a default Python
DEBUG Looking for exact match for request a default Python
DEBUG Searching for default Python interpreter in virtual environments, managed installations, or search path
DEBUG Searching for managed installations at `/home/user/.local/share/uv/python`
DEBUG Found managed installation `cpython-3.12.8-linux-x86_64-gnu`
DEBUG Found `cpython-3.12.8-linux-x86_64-gnu` at `/home/user/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/bin/python3.12` (managed installations)
DEBUG Using Python 3.12.8 interpreter at /home/user/.local/share/uv/python/cpython-3.12.8-linux-x86_64-gnu/bin/python3.12 for builds
DEBUG Using request timeout of 30s
DEBUG Solving with installed Python version: 3.12.8
DEBUG Solving with target Python version: >=3.12.8

and after this PR:

DEBUG Starting Python discovery for Python 3.11
DEBUG Looking for exact match for request Python 3.11
DEBUG Searching for Python 3.11 in virtual environments, managed installations, or search path
DEBUG Searching for managed installations at `/home/user/.local/share/uv/python`
DEBUG Skipping managed installation `cpython-3.12.8-linux-x86_64-gnu`: does not satisfy `3.11`
DEBUG Skipping managed installation `cpython-3.12.0-linux-x86_64-gnu`: does not satisfy `3.11`
DEBUG Skipping managed installation `cpython-3.10.15-linux-x86_64-gnu`: does not satisfy `3.11`
DEBUG Found `cpython-3.11.13-linux-x86_64-gnu` at `/usr/bin/python3.11` (first executable in the search path)
DEBUG Using Python 3.11.13 interpreter at /usr/bin/python3.11 for builds
DEBUG Using request timeout of 30s
DEBUG Solving with installed Python version: 3.11.13
DEBUG Solving with target Python version: >=3.11

and the output requirements.txt before:

# This file was autogenerated by uv via the following command:
#    uv pip compile requirements.in
certifi==2025.7.14
    # via requests
charset-normalizer==3.4.2
    # via requests
idna==3.10
    # via requests
markdown-it-py==3.0.0
    # via rich
mdurl==0.1.2
    # via markdown-it-py
pygments==2.19.2
    # via rich
requests==2.32.4
    # via -r requirements.in
rich==14.0.0
    # via -r requirements.in
urllib3==2.5.0
    # via requests

and after:

# This file was autogenerated by uv via the following command:
#    uv pip compile requirements.in
certifi==2025.7.14
    # via requests
charset-normalizer==3.4.2
    # via requests
idna==3.10
    # via requests
requests==2.32.4
    # via -r requirements.in
urllib3==2.5.0
    # via requests

This is also reflected in the unit tests.

@staticf0x staticf0x force-pushed the pip-compile-respect-python-version branch from 8fdd21d to 0507b50 Compare July 15, 2025 15:16
@notatallshaw
Copy link
Collaborator

Let's say my project supports Python 3.9+ but I build my project with Python 3.12, so I have a ".python-version" with 3.9 on the root and a ".python-version" with 3.12 in "build-project" sub directory.

Now because I pin my build requirements I call uv pip compile build-project/requirements.in -o build-project/requirements.txt in my build script in my Python 3.12 build environment.

This change will break that, so at the least it should be called out as a breaking change to pip-compile and the behavior should be documented.

The reason I have all these ".python-version" files floating around is to get dependendabot and other tools to know which Python is a good Python to default against, not to assist uv.

@zanieb
Copy link
Member

zanieb commented Jul 15, 2025

I think inferring the .python-version file to respect in uv pip compile might be too hard for users to reason about. It sounds complicated. If we do it, I don't think it should look for .python-version files relative to the input files. Just looking relative to the first one seems particularly arbitrary.

Now because I pin my build requirements I call uv pip compile build-project/requirements.in -o build-project/requirements.txt in my build script in my Python 3.12 build environment.

Wouldn't (or couldn't) we still prefer the current Python version in the environment over a .python-version file?

@notatallshaw
Copy link
Collaborator

Wouldn't (or couldn't) we still prefer the current Python version in the environment over a .python-version file?

You could, I would prefer it, but does it solve what OP wants? (Genuine question, I don't know).

@staticf0x
Copy link
Author

So we're running self-hosted Renovate with a Docker image with default Python 3.9 and while there are other Python versions available, because it's not the official Ubuntu based image, we can't install tools on the fly when processing repositories.

Now instead of relying on pip-compile, which is a very simple tool and relies on the correct Python being available in the venv, I was thinking: uv can install any Python version when you do uv sync on a freshly cloned project or when you change your .python-version, regardless on what is available on the target system, through the python-build-standalone project. This is great, I love this feature, but in #9581 we talk about this not working in uv pip compile.

The support for UV_PYTHON was added #11486 beginning with 0.6.0, but unlike uv sync the .python-version is not taken into account. So I naively thought I would add the support, can't be difficult right? But then I ran into the problem of multiple source files (described in the PR), uv potentially being invoked from outside of the project folder, etc...

Anyway, when you run Renovate with the pip-compile manager and your requirements.txt was compiled using uv pip compile, Renovate can cd into the directory and run uv pip compile requirements.in, so this is the reason why I tried to add support for reading .python-version in CWD. The reason for the sibling file lookup was to support invocations like uv pip compile project/dir/requirements.in for example. The .python-version file seems like the simplest, easily understandable solution for a lot of projects, but I admit I never thought about more complex situations where you have multiple such files.

I do realize this implementation is a bit flawed, so I definitely appreciate the discussion here. Hopefully my explanation avoids the XY problem and perhaps there's a way to solve my problem without this code change (that would be great though). Just mind that setting UV_PYTHON per repository is not possible for the Renovate use case.

And only now I realized I can pass the Python version in the compiled file header and Renovate will parse it and pass it back to uv pip compile, like so:

# This file was autogenerated by uv via the following command:
#    uv pip compile --python-version=3.12 --output-file=requirements.txt 

So this actually solves my use case, sorry for the confusion! I don't mind closing this PR, do you think this discussion is still helpful though? Or should we stick to #9581?

@zanieb
Copy link
Member

zanieb commented Jul 17, 2025

Thanks for all the details!

I'm glad just including it in the header solves the problem. That makes sense as a best practice to me anyway.

I think #9581 is the best place for future conversation on this, yeah. 

@zanieb zanieb closed this Jul 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants