Panic when installing package with pkg @ file:///path/to/local/pkg dependency

[abey79]

Checks

• I have checked that this issue has not already been reported.

• I have confirmed this bug exists on the latest version of pixi, using pixi --version.

Reproducible example

The issue can be reproduced with the following project: rerun.tar.gz

Uncompress in /tmp, clear the lock file, and pixi install:

❯ pixi install
⠙ updating pypi packages in 'default'
thread 'main' panicked at src/install_pypi.rs:199:14:
could not convert path into uv dist: NotFound(Url { scheme: "file", cannot_be_a_base: false, username: "", password: None, host: None, port: None, path: "/private/tmp/rerun/file:/private/tmp/rerun/clock", query: None, fragment: None })
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Issue description

Discussed in the following discord thread: https://discord.com/channels/1082332781146800168/1228365148243366010

The relevant pyproject is:

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[project]
name = "redirect"
version = "0.1.0"
requires-python = ">=3.8"
dependencies = [
  "clock @ file:///Users/hhip/src/rerun/examples/python/clock"
]

[tool.hatch.metadata]
# needed for hatch to be happy with file:// deps
allow-direct-references = true

Specifically, pixi seems to have an issue with @ file:///... dependency descriptor.

Expected behavior

The repro is a valid project and should work. In particular, uv is happy with it.

/tmp/rerun
❯ touch clock/clock.py   # missing from the repro archive

/tmp/rerun
❯ uv pip install redirect/.
Resolved 8 packages in 9ms
   Built clock @ file:///private/tmp/rerun/clock                                                                                                                                                        Downloaded 2 packages in 415ms
Installed 8 packages in 46ms
 + attrs==23.2.0
 + clock==0.1.0 (from file:///private/tmp/rerun/clock)
 + numpy==1.26.4
 + pillow==10.3.0
 + pyarrow==15.0.2
 + redirect==0.1.0 (from file:///private/tmp/rerun/redirect)
 + rerun-sdk==0.15.1
 + typing-extensions==4.11.0

[baszalmstra]

I think the error indicates that the file or directory at file:///Users/hhip/src/rerun/examples/python/clock does not exist.

Is this perhaps on a windows machine? Are you maybe missing the drive specifier?

[tdejager]

@baszalmstra No that's not it, it creates an erroneous path. Because of the double indirection in the project produces a 'file:://' in the wrong lock file variable. I've already investigated a bit but still need to solve!

[abey79]

A couple of thoughts that came up while poking at that stuff.

It may seem ad hoc/ugly to have full path hard-coded in dependencies. It turns out it can be a legitimate approach thanks to hatch's context formatting. Whit this feature, it's possible to have something like this as dependency:

dependencies = [
    "pkg @ {root:parent:uri}/pkg",
]

Hatch substitutes the {root:...:uri} pattern, so, from the pep 517 perspective, tooling sees this as a regular full path. Note that relative paths (e.g. @ file://../pkg) are not supported by pip nor uv.

Also, assuming pixi properly supports (indirect) local dependencies, it would be unfortunate that full paths made their way to the pixi.lock file. That would make it at best non-portable, at worst corrupt. It seems to me that properly supporting this use case would require translating full paths to project-relative, at least when the package in question are included in the project tree.

[tdejager]

Hmm for that case, I think we might need to make another exception if we want to support w.r.t locking.

I've made a fix for the repro here: #1196

Don't know if it is possible for you to give that PR a spin?

[tdejager]

We've discussed offline on discord. And this seems to resolve things, we are running into a different issue, but that's unrelated to this specific issue.

[davidbrochart]

I'm not sure it's the same issue, but I cannot use pixi with jupyter-collaboration:

git clone https://github.com/jupyterlab/jupyter-collaboration.git
cd jupyter-collaboration
pixi init

  × failed to parse project manifest
    ╭─[pyproject.toml:11:16]
 10 │     requires-python = ">=3.8"
 11 │ ╭─▶ dependencies = [
 12 │ │       # jupyter-server extensions
 13 │ │       "jupyter-server-ydoc @ {root:uri}/projects/jupyter-server-ydoc",
 14 │ │       # jupyterlab/notebook frontend extensions
 15 │ │       "jupyter-collaboration-ui @ {root:uri}/projects/jupyter-collaboration-ui",
 16 │ │       "jupyter-docprovider @ {root:uri}/projects/jupyter-docprovider",
 17 │ │       # the metapackage
 18 │ │       "jupyter-collaboration @ {root:uri}/projects/jupyter-collaboration",
 19 │ ├─▶ ]
    · ╰──┤ relative path without a working directory: {root:uri}/projects/jupyter-server-ydoc
    ·    │ jupyter-server-ydoc @ {root:uri}/projects/jupyter-server-ydoc
    ·    │                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 20 │     dynamic = ["version"]
    ╰────

[tdejager]

Yeah so its a related but not exactly the same issue, but that is because pep508-rs and the pyproject-toml-rs crates. Do not support these pep-508 specific extensions.

[tdejager]

To eleborate when reading the PyPA it is kind of implied that these file url's should be absolute only.

Even though requirements.txt files do support relative url's because they directly allow for specifying editables.

[tdejager]

@davidbrochart if you could make a seperate issue for your problem that would be great :)!