env: make pip/setuptools/wheel configurable

[abn]

This change introduces the following configuration options:

• virtualenvs.options.no-pip (Default: false)
• virtualenvs.options.no-wheel (Default: false)
• virtualenvs.options.no-setuptools (Default: false)

With these options, we allow, the user to control the inclusion of
pip, wheel and setuptools packages in new virtual environments.
The defaults used retain existing behaviour as of 1.1.5.

In addition the following now holds true.

• projects can define pip, wheel and setuptools as dependencies
• if any of these packages are in the project's dependency tree, poetry
  will manage their versions
• all poetry internal commands will use pip embedded in virtualenv
  irrespective of pip beig available in the environment
• if poetry run pip is executed, pip installed in the virtualenv will
  be prioritsed
• poetry install --remove-untracked will ignore these packages if
  they are not present in the project's and the options are not set to true dependency tree

Relates-to: #2826
Relates-to: #3916

Testing

Using pipx

pipx install --suffix=@3920 'poetry @ git+https://github.com/python-poetry/poetry.git@refs/pull/3920/head'
alias poetry=poetry@3920
poetry config virtualenvs.options.no-setuptools false
poetry lock
poetry run pip list
poetry add -D setuptools
poetry config virtualenvs.options.no-setuptools true
poetry install --remove-untracked
poetry run pip list

Using a container (podman | docker)

podman run --rm -i --entrypoint bash python:3.8 <<EOF
set -x
python -m pip install -q git+https://github.com/python-poetry/poetry.git@refs/pull/3920/head
poetry new foobar
pushd foobar
sed -i /pytest/d pyproject.toml
poetry config virtualenvs.options.no-setuptools false
poetry lock
poetry run pip list
poetry add -D setuptools
poetry config virtualenvs.options.no-setuptools true
poetry install --remove-untracked
poetry run pip list
EOF

[sdispater]

I am on the fence here since we are starting to leak implementation details in our own config which I don't necessarily like. The fact that we use virtual environments internally – and more precisely virtualenv – was always an implementation detail in my mind and the fact that we are exposing it is not the right direction, I think.

[abn]

I had to chew on this one a bit myself. I would have prefered this was not required and we used empty virtual environments by default. That works well, however we hit an issue IDEs start having issues since they expect setuptools to be available at the very least. Until that assumption is broken in the ecosystem we might be stuck with having to allow config to do this.

The way I see it, we do manage virtual environments whether it is via virtualenv, venv or something else in the future. This is not somehting that we can/will get away from. The configuration is quite generic in that it is disabling default packages (pip, setuptools, wheel). While the functionality maps direct to internals, it is still a poetry functionality.

The other option is to expose this in the poetry env use command or similar. Either way, I do not think we can get away from exposing these options. Happy to hear suggestions, if you have any in mind.

[abn]

Based on the conversation above we might revisit this later and instead move some of the changes into #4011. This will mean the current behavriour is not altered, however reserved packages are still allowed to be managed as a project dependency when required. If a user requires an empty virtual envrionment, they will be expected to create one themselves or remove these packages manually after creation.

[github-actions]

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.