Repository source precedence order issue

[elachere]

• I am on the latest Poetry version.

• I have searched the issues of this repo and believe that this is not a duplicate.

• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: Debian GNU/Linux bookworm/sid x86_64, Kernel: 5.10.0-2-amd64

• Poetry version: 1.1.13, 1.1.2, 1.1.3, 1.1.4

• Link of a Gist with the contents of your pyproject.toml file: https://gist.github.com/elachere/df77e06ed5b1830ff66c254919314c3d

Issue

I'm trying to install dependencies both from pypi and a private repo, here's my original pyproject.toml, following the official documentation:

[tool.poetry]
name = "project_name"
version = "0.1.0"
description = ""
authors = ["Me <[email protected]>"]

[[tool.poetry.source]]
name = "private"
url = "https://url/to/private_repo"
secondary = true

[tool.poetry.dependencies]
python = "^3.9"
Flask = "^2.1.2"
private_package = "*"

[tool.poetry.dev-dependencies]
black = "^22.6.0"
mypy = "^0.961

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

However, running poetry install results in poetry trying to install every requirement from the private repo, at least that's what I understand from the resulting traceback:

❯ poetry install
Updating dependencies
Resolving dependencies... (0.2s)

  RepositoryError

  403 Client Error: Forbidden for url: https://url/to/private_repo/mypy/

  at ~/.local/lib/python3.9/site-packages/poetry/repositories/legacy_repository.py:393 in _get
      389│             if response.status_code == 404:
      390│                 return
      391│             response.raise_for_status()
      392│         except requests.HTTPError as e:
    → 393│             raise RepositoryError(e)
      394│ 
      395│         if response.status_code in (401, 403):
      396│             self._log(
      397│                 "Authorization error accessing {url}".format(url=url), level="warn"

As you can see, it seems that poetry is trying to install mypy from https://url/to/private_repo/mypy/.

So far I have tried to:

• explicitly set the source for each dependency in the pyproject.toml file (e.g: mypy = {version = "^0.961", source = "pypi"}): then the same thing happens with subdependencies
• set the private repository url via the command poetry config repositories.private https://url/to/private_repo: does not seem to have any impact
• all possible combinations of the default and secondary keys for [[tool.poetry.source]]: nothing helps
• adding the private dependency with poetry add private_package --source private
• adding explicitly a source to official pypi and set it as default
• I clear the cache when trying something

N.B: the private repo I'm trying to install from does not require any kind of authentication ( I can install from it with pip install -i https://url/to/private_repo private_package)

Experiencing this behaviour with versions:

• 1.1.4
• 1.1.3
• 1.1.2

[dimbleby]

Per https://python-poetry.org/docs/master/repositories#secondary-package-sources

All package sources (including secondary sources) will be searched during the package lookup process.

I'm not sure that this is what people necessarily want: but it is the documented and expected behaviour.

I think that what a lot of people expect is that: if poetry finds a package in a primary repository, then it doesn't bother to look in any secondary repositories.

That wouldn't be very hard to arrange - relevant code is here - but probably trying to change it would provoke some discussion...

[chronossc]

I have hit this problem with s3pypi private repo. In our case when we access a non existent package in s3pypi repo, we receive 403, wich is a authentication problem. But, return 403 instead 404 can be a security measure for private repos in order to avoid bots to know which packages doesn't exist in repos.

[neersighted]

This is currently as designed -- see #6713 for a proposal to introduce the behavior you desire/expect.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.