Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow dependency name overriding or elimination #4422

Open
leorochael opened this issue Jun 20, 2024 · 16 comments
Open

Allow dependency name overriding or elimination #4422

leorochael opened this issue Jun 20, 2024 · 16 comments
Labels
enhancement New feature or request needs-design Needs discussion, investigation, or design

Comments

@leorochael
Copy link

leorochael commented Jun 20, 2024

While uv now has overrides, and issue #2686 is suggesting being able to declare certain overrides as pertaining to a specific dependency, I have an additional suggestion/request.

I would like to be able to override a package dependency with a package that has a different name.

As a concrete case, the FuelSDK package depends on suds-jurko, but has been tested to work with it's fork/successor project suds-community, a.k.a. just suds these days.

(Yes, the project has come full circle, with the fork of the fork officially taking over the original name).

A possibility that comes to mind would be a special syntax for the -o overrides.txt file like:

suds-jurko=>suds>=1.1,<2.0

Or perhaps:

suds-jurko:suds>=1.1,<2.0

Considering the long history of python packages that fork and continue the work of orphaned packages (e.g. PIL/Pillow), this could be a nice complement to the overrides functionality of uv.

Perhaps it could even be used to remove a dependency completely, e.g.:

  • suds-jurko=>!
  • suds-jurko=>
  • suds-jurko:!
  • suds-jurko:
@leorochael
Copy link
Author

Thinking about this some more, just the functionality of blocking a dependency from being installed already offers a way to do this kind of override.

That is, I could add to my requirements.in:

Fuel-SDK
suds>=1.1,<2.0

Then to my overrides one of the options for deleting a removing a dependency I mentioned previously, or perhaps just a special version. e.g one of:

  • suds-jurko==skip
  • suds-jurko==none
  • suds-jurko==

@zanieb zanieb added the enhancement New feature or request label Jun 20, 2024
@zanieb
Copy link
Member

zanieb commented Jun 20, 2024

Interesting idea! Makes some sense, though I'm also not quite sure how it should be expressed and if it's feasible.

@zanieb zanieb added the needs-design Needs discussion, investigation, or design label Jun 20, 2024
@leorochael leorochael changed the title Allow package name overriding Allow dependency name overriding or elimination Jun 21, 2024
@leorochael
Copy link
Author

Another option to express dependency elision is to use the URI version specifier to mean: install this package from nowhere.

  • suds-jurko@skip:
  • suds-jurko@about:blank

The last one was suggested to me by github copilot:

To explicitly reference nothing or create a URI that signifies an absence of a resource, you can use the about:blank URI. This URI leads to an empty document in web browsers and is often used to represent a null or non-existent resource in contexts where a URI is required but there is no actual resource to point to.

@leorochael
Copy link
Author

And since URIs effectively allow namespacing on schemes, and the definition of the interpretation of schemes is up to whoever introduces them, we could perhaps use schemes to implement the package replacement, e.g:

  • suds-jurko@replacement:suds>=1.1,<2.0

@leorochael
Copy link
Author

leorochael commented Jun 21, 2024

Or, if we want to follow examples of other tools which use prefixed schemes, like git+, we could namespace the scheme with uv+, like:

  • suds-jurko@uv+skip:
  • suds-jurko@uv+replacement:suds>=1.1,<2.0

@leorochael
Copy link
Author

Note that --no-emit-package/--unsafe-package suds-jurko doesn't help here, because suds-jurko is broken at package building, so resolution still fails.

@leorochael
Copy link
Author

An alternative that would help me would be to just ignore the sub-dependencies of a package on a per-package basis.

That is, instead of having an elision or renaming on my overrides file, I would like to add something like this in my requirements.in:

# FuelSDK has broken sub-dependencies:
--no-deps-for FuelSDK
FuelSDK
# Corrected FuelSDK dependencies manually added here:
pyjwt >= 1.5.3
requests >= 2.18.4
suds >= 1.1.2

For reference, such an option has been under discussion for a while in pypa/pip#9948

@charliermarsh
Copy link
Member

This seems like a reasonable use-case to me.

@hauntsaninja
Copy link
Contributor

hauntsaninja commented Jul 24, 2024

I think this would be very useful! At work, we have systems like this for first party packages, but not third party packages. It would make some things less painful if this were the case and unblock some other future use cases.

A related lower priority feature request would be the ability to override edges, not just nodes. E.g., if I know one package is overpinning a dependency in a way that I know is safe to override, I might still want to resolve older or error if something else pins it.

I don't have strong opinions on syntax. => is a good suggestion, but could be typo-ed. Maybe using @@ as a separator? lhs is the thing to be overriden, rhs is empty or PEP 508

node>=2
node @@ node>=2  # same as above
node @@ node_nextgen>=2  # replace node with node_nextgen
node @@  # eliminate node (and don't resolve its deps)
node -> dep @@ dep>=2  # override node's dependency on dep

If syntax is controversial, another option is to switch to TOML or something, especially if uv has plans to use such a format somewhere else.

@charliermarsh
Copy link
Member

I realized that this actually is possible today... You can use a never-truthy marker. For example, to remove typing-extensions, use an overrides.txt like:

typing-extensions ; sys_platform == 'never'

@charliermarsh
Copy link
Member

(This works because with overrides, we just replace all requirements of typing-extensions with whatever is in the overrides file.)

@leorochael
Copy link
Author

leorochael commented Jul 29, 2024

(This works because with overrides, we just replace all requirements of typing-extensions with whatever is in the overrides file.)

I'm surprised this works, because I'd expect it would result in the resolution being unsolvable...

And I'm slightly suspicious of relying on it and it being just accidental behaviour that might be "fixed" in the future...

@charliermarsh
Copy link
Member

@leorochael -- Can you say more about why you would expect that to be unsolvable? I would actually consider it a bug if the resolver failed there, rather than the other way around.

@leorochael
Copy link
Author

leorochael commented Jul 29, 2024

I might be reading it wrong, but for me an overrides file containing:

  • typing-extensions ; sys_platform == 'never'

Mean:

  • Only consider this typing-extensions version: the one that cannot be installed on your platform, because your platform is not never.

And if there are packages depending on typing-extensions, in my mind that would mean their dependencies would be unsolvable because of that.

To me, declaring a constraints override on an uninstallable package is different than declaring a constraints override on a package we're pretending is already installed.

@leorochael
Copy link
Author

leorochael commented Jul 29, 2024

Ah, but I see where my logic is failing.

The constraints override doesn't say: "depend on this uninstallable package".

It's saying: "when depending on typing-extensions, only do that if the platform is never", which never happens.

So, yeah, as long as we add a test for that and document it, then yes, that would already be supported.

@hauntsaninja
Copy link
Contributor

Oh, neat trick!

charliermarsh added a commit that referenced this issue Sep 18, 2024
## Summary

This PR enables users to provide pre-defined static metadata for
dependencies. It's intended for situations in which the user depends on
a package that does _not_ declare static metadata (e.g., a
`setup.py`-only sdist), and that is expensive to build or even cannot be
built on some architectures. For example, you might have a Linux-only
dependency that can't be built on ARM -- but we need to build that
package in order to generate the lockfile. By providing static metadata,
the user can instruct uv to avoid building that package at all.

For example, to override all `anyio` versions:

```toml
[project]
name = "project"
version = "0.1.0"
requires-python = ">=3.12"
dependencies = ["anyio"]

[[tool.uv.dependency-metadata]]
name = "anyio"
requires-dist = ["iniconfig"]
```

Or, to override a specific version:

```toml
[project]
name = "project"
version = "0.1.0"
requires-python = ">=3.12"
dependencies = ["anyio"]

[[tool.uv.dependency-metadata]]
name = "anyio"
version = "3.7.0"
requires-dist = ["iniconfig"]
```

The current implementation uses `Metadata23` directly, so we adhere to
the exact schema expected internally and defined by the standards. Any
entries are treated similarly to overrides, in that we won't even look
for `[email protected]` metadata in the above example. (In a way, this also
enables #4422, since you could remove a dependency for a specific
package, though it's probably too unwieldy to use in practice, since
you'd need to redefine the _rest_ of the metadata, and do that for every
package that requires the package you want to omit.)

This is under-documented, since I want to get feedback on the core ideas
and names involved.

Closes #7393.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request needs-design Needs discussion, investigation, or design
Projects
None yet
Development

No branches or pull requests

4 participants