1.2.0 fails to recognise external active virtualenv

[mmllee]

• Poetry version: 1.2.0
• Python version: 3.8.x
• OS version and name: ubuntu 22.04 and macOS 12.5
• pyproject.toml: any

• I am on the latest stable Poetry version, installed using a recommended method.
• I have searched the issues of this repo and believe that this is not a duplicate.
• I have consulted the FAQ and blog for any relevant entries or release notes.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option) and have included the output below.

Issue

External envs are not correctly detected by poetry 1.2.0

to reproduce (with default poetry config)

rm -rf /tmp/venv/ && python3 -m venv  /tmp/venv && . /tmp/venv/bin/activate && poetry install -v

shows:
Creating virtualenv ${poetry_project_name}-Wz7GVy6b-py3.8 in /Users/myuser/Library/Caches/pypoetry/virtualenvs

when
poetry config virtualenvs.create false is set
the above command produces:

Skipping virtualenv creation, as specified in config file.
Installing dependencies from lock file

Finding the necessary packages for the current system

Package operations: 286 installs, 16 updates, 0 removals

  • Updating six (1.15.0 /AppleInternal/Library/BuildRoots/a0876c02-1788-11ed-b9c4-96898e02b808/Library/Caches/com.apple.xbs/Sources/python3_modules/six-1.15.0-py2.py3-none-any.whl -> 1.16.0): Failed

  CalledProcessError

  Command '['/AppleInternal/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8', '/Users/myuser/Library/Application Support/pypoetry/venv/lib/python3.8/site-packages/virtualenv/seed/wheels/embed/pip-22.2.2-py3-none-any.whl/pip', 'install', '--disable-pip-version-check', '--prefix', '/AppleInternal/Library/Frameworks/Python.framework/Versions/3.8', '--upgrade', '--no-deps', '/Users/myuser/Library/Caches/pypoetry/artifacts/e8/bc/c6/8f0343a6a2046d1f17d0a28efec62de264c2eab04ebcb6b3a74d62a9f8/six-1.16.0-py2.py3-none-any.whl']' returned non-zero exit status 1.

as one can see poetry is trying to install to the system instead of the activated venv and fails (as expected) due to permissions

poetry env info doesn't show paths to the activated external venv:

➜  poetry env info

Virtualenv
Python:         3.8.9
Implementation: CPython
Path:           NA
Executable:     NA

System
Platform:   darwin
OS:         posix
Python:     3.8.9
Path:       /AppleInternal/Library/Frameworks/Python.framework/Versions/3.8
Executable: /AppleInternal/Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8

[neersighted]

I'm not sure what's going on here, but a clean reproduction in a container is needed.

For me, the following works as expected (on Poetry 1.2.1):

$ python -m venv /tmp/venv
$ . /tmp/venv/bin/activate
$ poetry install
$ poetry show
$ /tmp/venv/bin/pip list

poetry env info works as expected as well:

Virtualenv
Python:         3.10.7
Implementation: CPython
Path:           /tmp/venv
Executable:     /tmp/venv/bin/python
Valid:          True

System
Platform:   linux
OS:         posix
Python:     3.10.7
Path:       /usr
Executable: /usr/bin/python3.10

[mmllee]

FROM ubuntu:20.04 as dev
RUN apt-get update && apt-get install python3 python3-venv python3-pip curl
RUN curl -sSL https://install.python-poetry.org | python3 - --version=1.2.0
ADD pyproject.toml /project
RUN cd /project && python -m venv /tmp/venv && . /tmp/venv/bin/activate && ~/.local/bin/poetry install -v

// works for me both 1.2.0 and 1.2.1 as expected

[neersighted]

Its worth nothing that everything works for me as expected on macOS as well. I'm going to close this for now as neither of us can reproduce it reliably -- I suggest you inspect the shell environment after activate to make sure VIRTUAL_ENV and PATH are set correctly (local configuration is likely what is at fault for you here).

[john-sandall]

FWIW I've come across similar behaviour in both 1.2.0 and 1.2.1, and (in case it helps anyone else finding this thread) I fixed the issue by renaming the name key within [tool.poetry] section in pyproject.toml

• Poetry version: 1.2.1
• Python version: 3.10.4
• OS version and name: macOS 12.6

pyproject.toml looks like:

[tool.poetry]
name = "fsdl"

In my situation, I've created a virtualenv also called fsdl, activated it, and its in a folder called fsdl-text-recognizer-2022-labs. When I run poetry env info I get:

Virtualenv
Python:         3.10.5
Implementation: CPython
Path:           NA
Executable:     NA

System
Platform:   darwin
OS:         posix
Python:     3.10.5
Path:       /Users/john/.pyenv/versions/3.10.5
Executable: /Users/john/.pyenv/versions/3.10.5/bin/python3.10

If I change the name key in pyproject.toml to fullstackdeeplearning, and then rerun poetry env info, it gives

Virtualenv
Python:         3.10.4
Implementation: CPython
Path:           /Users/john/.virtualenvs/fsdl
Executable:     /Users/john/.virtualenvs/fsdl/bin/python
Valid:          True

System
Platform:   darwin
OS:         posix
Python:     3.10.4
Path:       /Users/john/.pyenv/versions/3.10.4
Executable: /Users/john/.pyenv/versions/3.10.4/bin/python3.10

This behaviour is reliable & consistent, i.e. change it back to fsdl, then poetry can't find the fsdl environment, change it to fullstackdeeplearning again and it pops up.

[neersighted]

@john-sandall you have a different issue -- please open a new issue if you can reproduce it in a container.

[neersighted]

(and please open a new issue if you have a reproduction for a closed issue, or a different manifestation with a different reproduction -- it's hard for people to see if you just comment on an existing (especially closed) issue)

[mmllee]

@neersighted sorry, but could it be that Poetry indeed has some bug that make it very fragile to discover if Venv is active or not ?

poetry/src/poetry/utils/env.py

Line 1689 in 93e23bc

# In this case we need to get sys.base_prefix

        # If base is None, it probably means this is
        # a virtualenv created from VIRTUAL_ENV.
        # In this case we need to get sys.base_prefix
        # from inside the virtualenv.

according to venv docs sys.prefix or sys.exec_prefix should be used instead (https://docs.python.org/3/library/venv.html)

When a virtual environment is active (i.e., the virtual environment’s Python interpreter is running), the attributes sys.prefix and sys.exec_prefix point to the base directory of the virtual environment, whereas sys.base_prefix and sys.base_exec_prefix point to the non-virtual environment Python installation which was used to create the virtual environment. If a virtual environment is not active, then sys.prefix is the same as sys.base_prefix and sys.exec_prefix is the same as sys.base_exec_prefix (they all point to a non-virtual environment Python installation)

[neersighted]

Our virtual environment detection mechanism is a bit complex as we do have a class-based Env that's used to do handling of different types of environment -- that being said, I'm not 100% sure what you mean.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.