Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[1.7] [Errno 2] No such file or directory: '/usr/pyvenv.cfg' when running poetry add in nvidia/cuda:...ubuntu22.04 image #8646

Closed
4 tasks done
CallumJHays opened this issue Nov 10, 2023 · 5 comments · Fixed by #8672
Labels
area/installer Related to the dependency installer kind/bug Something isn't working as expected

Comments

@CallumJHays
Copy link

CallumJHays commented Nov 10, 2023

  • I am on the latest stable Poetry version, installed using a recommended method.
  • I have searched the issues of this repo and believe that this is not a duplicate.
  • I have consulted the FAQ and blog for any relevant entries or release notes.
  • If an exception occurs when executing a command, I executed it again in debug mode (-vvv option) and have included the output below.

Issue

I'm running into a [Errno 2] No such file or directory: '/usr/pyvenv.cfg' error when trying to run poetry add or poetry install within a docker container based on nvidia/cuda:11.8.0-cudnn8-runtime-ubuntu22.04.

Debugging Details:

  • The issue does not occur in 1.6.1 - only 1.7.0
  • The issue does not occur with a different base image (ie FROM python:3.10)
    • clearly something in the nvidia image is causing the problems - but not sure what.
  • The issue does not occur unless poetry config virtual-envs.create false

I included a Dockerfile in my attached gist, with the build args POETRY_VERSION and POETRY_VIRTUALENVS_CREATE to help isolate the issue. The error can be demonstrated with the following command:

docker build . --build-arg POETRY_VERSION=1.7.0 --build-arg POETRY_VIRTUALENVS_CREATE=false

The docker build fails on the last step (RUN poetry add six -vvv), and produces the following output. PS: the package used to test (six) is arbitrary - it fails for any package I've tried

  > [7/7] RUN poetry add six -vvv:                                                                                                
0.452 Loading configuration file /root/.config/pypoetry/config.toml                                                              
0.463 Skipping virtualenv creation, as specified in config file.                                                                 
0.531                                                                                                                            
0.531   Stack trace:                                                                                                             
0.531 
0.531   17  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/cleo/application.py:327 in run
0.543        3250.543        326try:
0.543327exit_code = self._run(io)
0.543        328except BrokenPipeError:
0.543        329# If we are piped to another process, it may close early and send a
0.543 
0.543   16  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/console/application.py:190 in _run
0.549        188self._load_plugins(io)
0.549        1890.549190exit_code: int = super()._run(io)
0.549        191return exit_code
0.549        1920.549 
0.549   15  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/cleo/application.py:431 in _run
0.558        429io.input.interactive(interactive)
0.558        4300.558431exit_code = self._run_command(command, io)
0.558        432self._running_command = None
0.558        4330.558 
0.558   14  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/cleo/application.py:473 in _run_command
0.567        4710.567        472if error is not None:
0.567473raise error
0.567        4740.567        475return terminate_event.exit_code
0.567 
0.567   13  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/cleo/application.py:454 in _run_command
0.576        4520.576        453try:
0.576454self._event_dispatcher.dispatch(command_event, COMMAND)
0.576        4550.576        456if command_event.command_should_run():
0.576 
0.576   12  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/cleo/events/event_dispatcher.py:26 in dispatch
0.578         240.578         25if listeners:
0.57826self._do_dispatch(listeners, event_name, event)
0.578         270.578         28return event
0.578 
0.578   11  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/cleo/events/event_dispatcher.py:85 in _do_dispatch
0.579         83break
0.579         840.57985listener(event, event_name, self)
0.579         860.579         87def _sort_listeners(self, event_name: str) -> None:
0.579 
0.579   10  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/console/application.py:322 in configure_installer_for_event
0.585        320return
0.585        3210.585322cls.configure_installer_for_command(command, event.io)
0.585        3230.585        324│     @staticmethod
0.585 
0.585    9  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/console/application.py:329 in configure_installer_for_command
0.591        3270.591        328poetry = command.poetry
0.591329installer = Installer(
0.591        330io,
0.591        331command.env,
0.591 
0.591    8  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/installation/installer.py:74 in __init__
0.597         720.597         73if installed is None:
0.59774installed = self._get_installed()
0.597         750.597         76self._installed_repository = installed
0.598 
0.598    7  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/installation/installer.py:443 in _get_installed
0.604        4410.604        442def _get_installed(self) -> InstalledRepository:
0.604443return InstalledRepository.load(self._env)
0.604        4440.604 
0.604    6  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/repositories/installed_repository.py:274 in load
0.608        272continue
0.608        2730.608274package = cls.create_package_from_distribution(distribution, env)
0.608        2750.608        276if with_dependencies:
0.608 
0.608    5  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/repositories/installed_repository.py:112 in create_package_from_distribution
0.612        110return cls.create_package_from_pep610(distribution)
0.612        1110.612112is_standard_package = env.is_path_relative_to_lib(path)
0.612        1130.612        114source_type = None
0.612 
0.612    4  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/utils/env/virtual_env.py:154 in is_path_relative_to_lib
0.615        152def is_path_relative_to_lib(self, path: Path) -> bool:
0.615        153return super().is_path_relative_to_lib(path) or (
0.615154self.includes_system_site_packages
0.615        155and SystemEnv(Path(sys.prefix)).is_path_relative_to_lib(path)
0.615        156│         )
0.615 
0.615    3  /usr/lib/python3.10/functools.py:981 in __get__
0.628        979val = cache.get(self.attrname, _NOT_FOUND)
0.628        980if val is _NOT_FOUND:
0.628981val = self.func(instance)
0.628        982try:
0.628        983cache[self.attrname] = val
0.628 
0.628    2  ~/.local/share/pypoetry/venv/lib/python3.10/site-packages/poetry/utils/env/virtual_env.py:146 in includes_system_site_packages
0.631        144re.search(
0.631        145r"^\s*include-system-site-packages\s*=\s*true\s*$",
0.631146pyvenv_cfg.read_text(),
0.631        147re.IGNORECASE | re.MULTILINE,
0.631        148│             )
0.631 
0.631    1  /usr/lib/python3.10/pathlib.py:1134 in read_text
0.651        1132"""
0.651        1133│         encoding = io.text_encoding(encoding)
0.651      → 1134│         with self.open(mode='r', encoding=encoding, errors=errors) as f:
0.651        1135│             return f.read()
0.651        1136│ 
0.651 
0.651   FileNotFoundError
0.651 
0.651   [Errno 2] No such file or directory: '/usr/pyvenv.cfg'
0.651 
0.651   at /usr/lib/python3.10/pathlib.py:1119 in open
0.670       1115│         the built-in open() function does.
0.670       1116│         """
0.670       1117if "b" not in mode:
0.670       1118encoding = io.text_encoding(encoding)
0.6701119return self._accessor.open(self, mode, buffering, encoding, errors,
0.670       1120newline)
0.670       11210.670       1122def read_bytes(self):
0.670       1123│         """
------
Dockerfile:18
--------------------
  16 |     
  17 |     COPY pyproject.toml .
  18 | >>> RUN poetry add six -vvv
--------------------
ERROR: failed to solve: process "/bin/sh -c poetry add six -vvv" did not complete successfully: exit code: 1
@CallumJHays CallumJHays added kind/bug Something isn't working as expected status/triage This issue needs to be triaged labels Nov 10, 2023
@CallumJHays CallumJHays changed the title [Errno 2] No such file or directory: '/usr/pyvenv.cfg' when running poetry add in nvidia/cuda:...ubuntu22.04 image [1.7] [Errno 2] No such file or directory: '/usr/pyvenv.cfg' when running poetry add in nvidia/cuda:...ubuntu22.04 image Nov 10, 2023
@hanslovsky
Copy link

hanslovsky commented Nov 10, 2023

I have a similar issue using a Python interpreter in a conda environment for poetry. When I first run

python -m poetry lock

Everything works as expected, but for subsequent calls to the same command, I have to delete and re-create my conda environment. I am not 100% sure that this is deterministic or what causes the issue.

  • Poetry version: 1.7.0
  • Python version: 3.10.13
  • OS version and name: macOS Ventura 13.5.1 (22G90)
  • pyproject.toml: I will try to build a MWE and share
  • environment.yaml:
    channels:
     - conda-forge
     - nodefaults
    dependencies:
     - python=3.10  # TODO: rapids supports only python 3.9. and 3.10 for installation via pip at this time
     - pip
     - poetry

This is the error that I see:

[Errno 2] No such file or directory: '$HOME/mambaforge/envs/<env-name>/pyvenv.cfg'

@AbdelrhmanHamouda
Copy link

I have a somewhat similar issue. poetry works fine on terminal, however, as soon as it is called from a dockerfile, it fails directly in the install step!

This only started happening with poetry 1.7.0

dickerfile

ARG PYTHON_VERSION=3.11
FROM python:$PYTHON_VERSION-slim AS ci

# Keeps Python from generating .pyc files in the container
ENV PYTHONDONTWRITEBYTECODE=1

# Turns off buffering for easier container logging
ENV PYTHONUNBUFFERED=1

# Install needed utlities and compilers that may be required for certain packages or platforms.
RUN  : \
     && apt-get update \
     && DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
        git \
        curl \
        bash \
        vim \
        build-essential \
     && rm -rf /var/lib/apt/lists/*

# Update pip to the latest available version
RUN pip3 install --upgrade pip

# Set the working directory in the container
WORKDIR /ciUser

# Set python path
ENV  PYTHONPATH=/ciUser

# Install & configure needed package
RUN : \
    && pip3 install poetry \
    && poetry config virtualenvs.create false

# Install requirements
COPY pyproject.toml .
COPY poetry.lock .
RUN poetry install

# Init git repo
RUN git init -b ci

# Port commit checks config
COPY .pre-commit-config.yaml .

# Port flake8 config
COPY .flake8 .

# Install & pre-load pre-commit enviroments
RUN : \
    && pre-commit install --install-hooks \
    && pre-commit run

# Copy the content of the `src` directory to the working directory
COPY src/ ./src

# Copy template information
COPY .cruft.json .

# Copy `README.md` to the working directory
COPY README.md ./README.md

# Add all files to trigger commit checks
RUN git add .

# Enable container to be used as a binary that accepts input
ENTRYPOINT ["/bin/bash", "-c"]

error:

>  • Installing pdoc (12.3.1)
>
>   FileNotFoundError
>
>   [Errno 2] No such file or directory: '/usr/local/lib/python3.11/site-packages/virtualenv/activation/nushell/activate.nu'
>
>   at /usr/local/lib/python3.11/pathlib.py:1044 in open
>       1040│         the built-in open() function does.
>       1041│         """
>       1042│         if "b" not in mode:
>       1043│             encoding = io.text_encoding(encoding)
>     → 1044│         return io.open(self, mode, buffering, encoding, errors, newline)
>       1045│
>       1046│     def read_bytes(self):
>       1047│         """
>       1048│         Open the file in bytes mode, read it, and close the file.

@dimbleby
Copy link
Contributor

via #8359 @abompard

@radoering radoering added area/installer Related to the dependency installer and removed status/triage This issue needs to be triaged labels Nov 14, 2023
@hanslovsky
Copy link

I can confirm that my issue above is now resolved, using the latest commit on master b9aab34

Copy link

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Feb 29, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area/installer Related to the dependency installer kind/bug Something isn't working as expected
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants