uv pip compile chokes on requirements files with explicit file:... requirements

[Julian]

Given a trivial (i.e. empty) package foo, and a requirements.in containing:

file:.#egg=foo

(i.e. simply saying "install this package"), uv pip compile requirements.in crashes with:

⊙  uv cache clean && uv pip compile --verbose requirements.in                                                                                                                                                                                                                   julian@Airm
Clearing cache at: /Users/julian/Library/Caches/uv
Removed 578 files (19.3MiB)
DEBUG Starting interpreter discovery for any Python
DEBUG Looking for exact match for request any Python
DEBUG Searching for Python interpreter in all sources
DEBUG Found CPython 3.12.3 at `/Users/julian/.local/bin/python3` (search path)
DEBUG Using Python 3.12.3 interpreter at /opt/homebrew/opt/[email protected]/bin/python3.12 for builds
DEBUG Using registry request timeout of 30s
DEBUG Acquired lock for `/Users/julian/Library/Caches/uv/built-wheels-v3/path/2d0737fc0df579d6`
error: Failed to read from the distribution cache
  Caused by: failed to query metadata of file `/Users/julian/Desktop/foo/.#egg=foo`
  Caused by: No such file or directory (os error 2)

pip-compile of course still works:

⊙  pip-compile --strip-extras requirements.in                                                                                                                                                                                                                                   julian@Airm
#
# This file is autogenerated by pip-compile with Python 3.12
# by the following command:
#
#    pip-compile --strip-extras requirements.in
#
file:.#egg=foo
    # via -r requirements.in

though uv is unable to install what it produces, even though pip can:

⊙  uv venv venv                                                                                                                                                                                                                                                                 julian@Airm
Using Python 3.12.3 interpreter at: /opt/homebrew/opt/[email protected]/bin/python3.12
Creating virtualenv at: venv
Activate with: source venv/bin/activate

~/Desktop/foo is a git repository on main 
⊙  uv pip install --python venv/bin/python -r requirements.txt                                                                                                                                                                                                                  julian@Airm
error: Failed to read from the distribution cache
  Caused by: failed to query metadata of file `/Users/julian/Desktop/foo/.#egg=foo`
  Caused by: No such file or directory (os error 2)

~/Desktop/foo is a git repository on main 
⊙  python3.12 -m venv venv-venv                                                                                                                                                                                                                                                 julian@Airm
venv-ven%                                                                                                                                                                                                                                                                                   
~/Desktop/foo is a git repository on main 
⊙  venv-venv/bin/python -m pip install -r requirements.txt                                                                                                                                                                                                                      julian@Airm
Processing /Users/julian/Desktop/foo (from -r requirements.txt (line 7))
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: foo
  Building wheel for foo (pyproject.toml) ... done
  Created wheel for foo: filename=foo-0.1.dev1+g25e84d5-py3-none-any.whl size=2679 sha256=3185e32701885b76635d347608f2c457e39d2ce15c05099bc1c5100f47517696
  Stored in directory: /private/var/folders/8x/h479j0p15cq6l3m2r26kfdkm0000gn/T/pip-ephem-wheel-cache-o_zbhzsx/wheels/8b/19/c8/73a63a20645e0f1ed9aae9dd5d459f0f7ad2332bb27cba6c0f
/Users/julian/Desktop/foo/venv-venv/lib/python3.12/site-packages/pip/_internal/metadata/importlib/_dists.py:73: DeprecationWarning: Unimplemented abstract methods {'locate_file'}
  return cls(files, info_location)
Successfully built foo
Installing collected packages: foo
Successfully installed foo-0.1.dev1+g25e84d5

This is on macOS with:

⊙  uv --version                                                                                                                                                                                                                                                                 julian@Airm
uv 0.2.5

where it seems some parsing is being done but the fragment hasn't been stripped.

(#3180 is related, as it's there again that this manifests for me, I once again cannot output something which both uv and pip are happy with, being on an older uv when first running the compile managed to get a compiled requirements file which as above uv itself could then not install)

[charliermarsh]

We can strip it, but why include the egg fragment at all?

[Julian]

That's the standard (to pip, it's not in any PEP AFAIK) way to indicate what the name of your package is. In which cases at this point in 2024 is that needed rather than falling back to parsing it out of pyproject.toml once building the wheel starts I don't know, but as far as I recall it's always been encouraged to include.

[Julian]

https://pip.pypa.io/en/stable/cli/pip_install/#working-out-the-name-and-version is the pip docs though they don't elaborate any further really.

[charliermarsh]

I'll fix this case but note: https://pip.pypa.io/en/stable/topics/vcs-support/#url-fragments

pip also looks at the egg fragment specifying the “project name”. In practice the egg fragment is only required to help pip determine the VCS clone location in editable mode. In all other circumstances, the egg fragment is not necessary and its use is discouraged.

[Julian]

That section seems to only be referring to VCS URLs, though I'm sure you're right to point it out as it seems unlikely that it should be any more needed for archive or path URLs -- but given pypa/pip#1289 still being open and that more broadly as an end user it's been way easier over the years to find an incantation that works and never change rather than keep track of what's what, I'm quite glad that you're interested in the fix nonetheless.

[charliermarsh]

My read of that thread and the Python Discuss thread is that it's only necessary at all for editable VCS URLs though I don't fully understand why it's necessary even in that case, and that the goal is to deprecate it entirely. I only mention this because I don't want others to read this issue and think that #egg is encouraged either here or in pip.

Regardless, fixed in #3940, since it's a legitimate bug on our end!