tweak the order of index priority #2083
Conversation
This removes all public methods on `IndexUrls` except for `indexes`. The `index` and `extra_index` methods weren't being used anywhere else, so those were just un-exported. The `no_index` method was being used to check if the `--no-index` parameter was provided, but we can actually encapsulate that by simply not returning any indexes when it's set. I did this so that callers don't need to know whether `no_index` is enabled or not. They just check the iterator given to them instead.
Previously, we would prioritize `--index-url` over all `--extra-index-url` values. But now, we prioritize all `--extra-index-url` values over `--index-url`. That is, `--index-url` has gone from the "primary" index to the "fallback" index. In most setups, `--index-url` is left as its default value, which is PyPI. The ordering of `--extra-index-url` with respect to one another remains the same. That is, in `--extra-index-url foo --extra-index-url bar`, `foo` will be tried before `bar`. Finally, note that this specifically does not match `pip`'s behavior. `pip` will attempt to look at versions of a package from all indexes in which in occurs. `uv` will stop looking for versions of a package once it finds it in an index. That is, for any given package, `uv` will only utilize versions of it from a single index. Ref #171, Fixes #1377, Fixes #1451, Fixes #1600
Since `black 23.10.1` isn't on test.pypi.org (but some versions of `black` are) and since --extra-index-url now takes priority, this command was failing. Since this is just testing that our index URLs are written out when requested, we flip the order of the indexes to get the previous behavior.
BurntSushi
force-pushed
the
ag/multi-registry
branch
from
212a995
to
323bae5
Compare
|
I believe this helps with dependency confusion attacks. If I'm providing an extra index URL that's usually the index that I control. The attack would be that someone uploads a package with the same name to the primary index (i.e. PyPI) and suddenly we use that package instead of the one I intend to. |
As a counter-point (not suggesting this is the most common) - we use extra-index-url to install a vendor library (we pay them for access), and the index is horribly slow. |
|
I just ran into an issue which is caused by this change. #15 0.746 Creating virtualenv at: /app/.venv
#15 0.775 uv::venv::seed
#15 0.775
#15 0.775 × Failed to install seed packages
#15 0.775 ├─▶ No solution found when resolving: pip, setuptools, wheel
#15 0.775 ├─▶ Failed to download and build: pip==20.2.4
#15 0.775 ╰─▶ Building source distributions is disabledAfter digging a bit, and trying to understand why this old Is there any plan to mitigate this issue ? Thanks ! |
|
For clarity, you can always revert to the previous behavior by inverting the order: This would tell uv to look in PyPI first, and then only fall back to your index if a package doesn't exist on PyPI. |
Previously,
uvwould always prioritize the index given by--index-url. It would then try any indexes after that given by zeroor more
--extra-index-urlflags. This differed frompipin that anypriority was given at all, where
pipdoesn't guarantee any priorityordering of indexes.
We could go in the direction of mimicing
pip's behavior here, but itat present has issues with dependency confusion attacks where packages
may get installed from indexes you don't control. More specifically,
there is an issue of different trust levels. See discussion in #171 and
PEP-0708 for more on the security impact.
In contrast,
uvwill only select versions for a package from a singleindex. That is, even if
foois in indexesaandb, it willonly consider the versions from the index that it checks first. This
probably helps with respect to dependency confusion attacks, but also
means that
uvdoesn't quite cover all of the same use cases aspip.In this PR, we retain the notion of prioritizing indexes, but
tweak it so that PyPI is preferred last as opposed to first. Or
more precisely, the
--index-urlflag specifies a fallback index,not the primary index, and is deprioritized beneath every index
specified by
--extra-index-url. The ordering among indexes given by--extra-index-urlremains the same: earlier indexes are prioritizedover later indexes.
While this tweak likely won't hit all use cases, I believe it will
resolve some of the most common pain points without exacerbating
dependency confusion problems.
Ref #171, Fixes #1377, Fixes #1451, Fixes #1600