Installing packages fails when already cached on Windows (making 1.1.9+ unusable on Windows)

[bparzella]

• I am on the latest Poetry version.
• I have searched the issues of this repo and believe that this is not a duplicate.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: Windows 10
• Poetry version: 1.1.9-1.1.11
• Link of a Gist with the contents of your pyproject.toml file:

name = "poetry_test"
version = "0.1.0"
description = ""
authors = ["Your Name <[email protected]>"]

[tool.poetry.dependencies]
python = "^3.6"
lxml = "^4.6.3"

[tool.poetry.dev-dependencies]

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

Issue

Installing a package that is already in the local pypoetry cache on windows fails since poetry 1.1.9.

$ poetry install -vvv --no-root
Using virtualenv: D:\user\src\temp\poetry_test\poetry_env
Installing dependencies from lock file

Finding the necessary packages for the current system

Package operations: 1 install, 0 updates, 0 removals

  • Installing lxml (4.6.3): Pending...
  • Installing lxml (4.6.3): Failed

  ValueError

  File \C:\Users\user\AppData\Local\pypoetry\Cache\artifacts\f7\90\e7\2d9752b05431b2ba90b4626b90f1674c505301a7cdbab89fd620aee824\lxml-4.6.3-cp36-cp36m-win_amd64.whl does not exist

  at D:\user\src\temp\poetry_test\poetry_env\lib\site-packages\poetry\core\packages\file_dependency.py:40 in __init__
       36│             except FileNotFoundError:
       37│                 raise ValueError("Directory {} does not exist".format(self._path))
       38│
       39│         if not self._full_path.exists():
    →  40│             raise ValueError("File {} does not exist".format(self._path))
       41│
       42│         if self._full_path.is_dir():
       43│             raise ValueError("{} is a directory, expected a file".format(self._path))
       44│

The problem does not occur, if the package is not cached.

Python version used is 3.6.8

Workaround

As a workaround we downgraded to 1.1.8 and poetry-core 1.0.4.

IMHO

It seems like this issue comes from the commit 8238cab

The call to "archive.path" in line 615 of 'poetry/installation/executor.py' seems to remove only two of the three slashes for Windows file URLs (file:///c:/.....). This results in an invalid path with a leading slash (visible in the command output File \C:\Users\user\AppData), which in the end leads to a missing file exception.

[klnrdknt]

I believe this is the same issue #4479

[bparzella]

The basic issue seems to be the same, but #4479 is about about the pre-release version 1.2.0a1 from the beginning of august.

Now the latest stable release 1.1.9 (september 18th) is affected which is supposed to be run on productive systems and makes it pretty much unusable on windows (at least in the conditions we are using it).

This information is missing in #4479 IMHO.

[klnrdknt]

Could this be related to the poetry-core update? Found a couple of issues with similar problems.

#4085

[bparzella]

That I don't know.

[Suween]

The workaround works. It seems that 1.1.8 is not compatible with core 1.0.5, but if you pip install==1.1.8, it comes with peotry core 1.0.5.

[jnoelvictorino]

Workaround works at 1.1.8

[DavisDmitry]

I have a similar problem when poetry tries to install colorama

Package operations: 10 installs, 0 updates, 0 removals

  • Installing colorama (0.4.4)

  ValueError

  File \C:\Users\Admin\AppData\Local\pypoetry\Cache\artifacts\9e\b3\11\7d87ac44fdb2d557301f1f4086a37c080d1482a98751abe7cdbabbad26\colorama-0.4.4-py2.py3-none-any.whl does not exist

  at ~\.poetry\lib\poetry\_vendor\py3.9\poetry\core\packages\file_dependency.py:40 in __init__
       36│             except FileNotFoundError:
       37│                 raise ValueError("Directory {} does not exist".format(self._path))
       38│
       39│         if not self._full_path.exists():
    →  40│             raise ValueError("File {} does not exist".format(self._path))
       41│
       42│         if self._full_path.is_dir():
       43│             raise ValueError("{} is a directory, expected a file".format(self._path))
       44│

Windows 10 19043.1237
Python 3.9.7
Poetry 1.1.10
Poetry-core 1.0.6

[mario-bermonti]

I am having the same issue on github’s CI Windows machine.

[bparzella]

Tested with 1.1.10, same problem.

A pull request that seems to fix this issue: #4531

[mario-bermonti]

The work around worked for me, but I had to install poetry-core manually (poetry-core==1.0.4)

[thearchitector]

Uninstalling poetry, removing all poetry folders and files, then reinstalling, also worked for the initial poetry install/add/remove command (but not any latter ones).

The workaround resolved the issue across the board for me as well.

[Ivoz]

Still borked with 1.1.11 on windows, anything from the Cache fails. So stable poetry is pretty stuffed on Windows atm

[serverwentdown]

Will be fixed by #4531 (1.2) and #4549 (1.1)

[serverwentdown]

If any of you have some free time, can you help me run these tests on Windows without the fix applied, and with the fix applied?

[serverwentdown]

@rubendibattista I don't think so, that issue seems to be due to some other reason, not Paths starting with slashes.

[Falmarri]

I'm surprised this is allowed to go on for over a month. This makes poetry 100% unusable on windows. In fact, even downgrading doesn't help if you're on the newest version of python because there's a fix for that in poetry 1.1.10.

[saftanas]

I have this issue and need to constantly delete my ~\AppData\Local\pypoetry\Cache\artifacts folder to fix the issue when updating dependencies. Any update on whether or not a release will come soon that will have a patch for this issue? (using 1.1.11)

[Diggsey]

For those with this issue, the following fix worked for me:

• Open ~\.poetry\lib\poetry\_vendor\py3.8\poetry\core\packages\utils\link.py (may need to change python version)
• Add import urllib.request at the top
• Modify Link.path to look like the following:

    @property
    def path(self):  # type: () -> str
        res = urlparse.urlsplit(self.url)
        if res.scheme == 'file':
            netloc = res.netloc
            if netloc:
                netloc = '\\\\' + netloc
            return urllib.request.url2pathname(netloc + res.path)
        else:
            return urlparse.unquote(res.path)

This seems quite a bit simpler than the "fix" PR?

[pohlt]

I still see exactly this issue with Poetry 1.2.0a2. Is this expected?

[serverwentdown]

@pohlt Wait a while for the next release. In the meantime, you can install the 1.2 branch with:

(Invoke-WebRequest -Uri https://raw.githubusercontent.com/python-poetry/poetry/master/install-poetry.py -UseBasicParsing).Content | python - --git git+https://github.com/python-poetry/poetry.git@master

[lars-reimann]

@pohlt Wait a while for the next release. In the meantime, you can install the 1.2 branch with:

(Invoke-WebRequest -Uri https://raw.githubusercontent.com/python-poetry/poetry/master/install-poetry.py -UseBasicParsing).Content | python - --git git+https://github.com/python-poetry/poetry.git@master

This gives me an error

ERROR: Invalid requirement: 'git+git+https://github.com/python-poetry/poetry.git@master' 

Removing the git+ works, though:

(Invoke-WebRequest -Uri https://raw.githubusercontent.com/python-poetry/poetry/master/install-poetry.py -UseBasicParsing).Content | python - --git https://github.com/python-poetry/poetry.git@master

[teucer]

@neersighted @serverwentdown Seems that the PR has been merged. When can we expect a new release?

[Ivoz]

1.1.12 is released with this fix

[mohghaderi]

Same problem exists with Poetry==1.3.2 and poetry-core==1.4.0 installed inside conda env. Windows 11 (and 10), Python=3.9.7 (Github Actions, Miniconda)

CalledProcessError

  Command 'C:\Miniconda3\envs\repo-name\python.exe -m pip install --disable-pip-version-check --isolated --no-input --prefix C:\Miniconda3\envs\repo-name --upgrade --no-deps C:\Users\runneradmin\AppData\Local\pypoetry\Cache\artifacts\eb\e4\7b\0856e7bf2096007a4407c92df58e703b0b78c2b5c6d63ef9d18d23d724\charset_normalizer-3.0.1-cp39-cp39-win_amd64.whl' returned non-zero exit status 1.

  at C:\Miniconda3\envs\repo-name\lib\subprocess.py:528 in run
       524|             # We don't call process.wait() as .__exit__ does that for us.
       525|             raise
       526|         retcode = process.poll()
       527|         if check and retcode:
    >  528|             raise CalledProcessError(retcode, process.args,
       529|                                      output=stdout, stderr=stderr)
       530|     return CompletedProcess(process.args, retcode, stdout, stderr)
       531| 
       532| 

The following error occurred when trying to handle this error:


  EnvCommandError

  Command C:\Miniconda3\envs\repo-name\python.exe -m pip install --disable-pip-version-check --isolated --no-input --prefix C:\Miniconda3\envs\repo-name --upgrade --no-deps C:\Users\runneradmin\AppData\Local\pypoetry\Cache\artifacts\eb\e4\7b\0856e7bf2096007a4407c92df58e703b0b78c2b5c6d63ef9d18d23d724\charset_normalizer-3.0.1-cp39-cp39-win_amd64.whl errored with the following return code 1, and output: 
  Processing c:\users\runneradmin\appdata\local\pypoetry\cache\artifacts\eb\e4\7b\0856e7bf2096007a4407c92df58e703b0b78c2b5c6d63ef9d18d23d724\charset_normalizer-3.0.1-cp39-cp39-win_amd64.whl

  Installing collected packages: charset-normalizer

    Attempting uninstall: charset-normalizer

      Found existing installation: charset-normalizer 2.1.1

      Uninstalling charset-normalizer-2.1.1:

        Successfully uninstalled charset-normalizer-2.1.1

  ERROR: Could not install packages due to an OSError: [WinError 5] Access is denied: 'C:\\Users\\runneradmin\\AppData\\Local\\Temp\\pip-uninstall-hvwco4b4\\normalizer.exe'

  Consider using the `--user` option or check the permissions.

  

  

  at C:\Miniconda3\envs\repo-name\lib\site-packages\poetry\utils\env.py:1540 in _run
      1536|                 output = subprocess.check_output(
      1537|                     command, stderr=subprocess.STDOUT, env=env, **kwargs
      1538|                 )
      1539|         except CalledProcessError as e:
    > 1540|             raise EnvCommandError(e, input=input_)
      1541| 
      1542|         return decode(output)
      1543| 
      1544|     def execute(self, bin: str, *args: str, **kwargs: Any) -> int:

The following error occurred when trying to handle this error:


  PoetryException

  Failed to install C:/Users/runneradmin/AppData/Local/pypoetry/Cache/artifacts/eb/e4/7b/0856e7bf2096007a4407c92df58e703b0b78c2b5c6d63ef9d18d23d724/charset_normalizer-3.0.1-cp39-cp39-win_amd64.whl

  at C:\Miniconda3\envs\repo-name\lib\site-packages\poetry\utils\pip.py:58 in pip_install
       54| 
       55|     try:
       56|         return environment.run_pip(*args)
       57|     except EnvCommandError as e:
    >  58|         raise PoetryException(f"Failed to install ***path.as_posix()***") from e
       59| 

Error: Process completed with exit code 1.

any suggestions?

[neersighted]

That is a completely different issue as evidenced by the stack trace; can you please open a new issue with reproduction instructions (e.g. a pyproject.toml tested to reliably reproduce + version information)?

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.