Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[1.1.0b2] Bumping version leaves orphaned dist-info in site-packages #2918

Closed
3 tasks done
MrGreenTea opened this issue Sep 15, 2020 · 16 comments · Fixed by #3900
Closed
3 tasks done

[1.1.0b2] Bumping version leaves orphaned dist-info in site-packages #2918

MrGreenTea opened this issue Sep 15, 2020 · 16 comments · Fixed by #3900
Labels
kind/bug Something isn't working as expected

Comments

@MrGreenTea
Copy link
Contributor

MrGreenTea commented Sep 15, 2020

Issue

When updating my packages version with poetry version old dist-info directories are left in site-packages.
This trips up pip, importlib and pkg_resources for example when trying to find the packages information.
To reproduce:
(run with poetry config virtualenvs.in-project true)

  1. ❯ poetry new example
  2. ❯ poetry install
  3. ❯ ls .venv/lib/python3.8/site-packages/example*.dist-info
.venv/lib/python3.8/site-packages/example-0.1.0.dist-info:
INSTALLER  METADATA  RECORD
  1. ❯ poetry version minor && poetry install
  2. ❯ ls .venv/lib/python3.8/site-packages/example*.dist-info
.venv/lib/python3.8/site-packages/example-0.1.0.dist-info:
INSTALLER  METADATA  RECORD

.venv/lib/python3.8/site-packages/example-0.2.0.dist-info:
INSTALLER  METADATA  RECORD
  1. ❯ poetry run pip show example
Name: example
Version: 0.2.0
Summary: 
Home-page: None
Author: Jonas Bulik
Author-email: [email protected]
License: None
Location: /tmp/example/.venv/lib/python3.8/site-packages
Requires: 
Required-by: 
  1. ❯ poetry version 0.1.0
  2. ❯ poetry run pip show example
Name: example
Version: 0.2.0
Summary: 
Home-page: None
Author: Jonas Bulik
Author-email: [email protected]
License: None
Location: /tmp/example/.venv/lib/python3.8/site-packages
Requires: 
Required-by: 
@MrGreenTea MrGreenTea added kind/bug Something isn't working as expected status/triage This issue needs to be triaged labels Sep 15, 2020
@finswimmer
Copy link
Member

Hello @MrGreenTea,

I guess #2855 is related, is it?

fin swimmer

@MrGreenTea
Copy link
Contributor Author

Hello @MrGreenTea,

I guess #2855 is related, is it?

fin swimmer

@finswimmer Looks like the same cause, yeah.

@jkbecker
Copy link

jkbecker commented Feb 2, 2021

I can reproduce that, and problematically, version bumping will

  • always report the newest version as being installed during poetry install
  • however, the actual code being run may or may not be the newest version

For example (excerpt from a discussion I had with sinoroc on the Discord trying to debug version issues of my project):
image

Is this currently being worked on somehow, or is there a clean workaround in the meantime?

@sinoroc
Copy link

sinoroc commented Feb 3, 2021

Does running poetry run python -m pip uninstall YourProjectName before running poetry install again help? It might be necessary to run the pip uninstall step more than one time until all old dist-info directories are removed. Otherwise removing the old dist-info directories manually seems like a good workaround. Another thing would be to delete the virtual environment and recreate it from scratch.

@ns-admetrics
Copy link

Using pip uninstall has the issue that you don't know how often to run it (potentially there are multiple old versions), but pip uninstall does not return an error code if there was no such package.

@fredrikaverpil
Copy link
Contributor

@ns-admetrics

Using pip uninstall has the issue that you don't know how often to run it (potentially there are multiple old versions), but pip uninstall does not return an error code if there was no such package.

It seems however, that you will get a certain error message whenever pip has removed all the different versions:

$ pip uninstall b
Found existing installation: b 0.0.5
Uninstalling b-0.0.5:
  Would remove:
    /home/fredrik/code/repos/a/.venv/bin/bclix
    /home/fredrik/code/repos/a/.venv/lib/python3.9/site-packages/b-0.0.5.dist-info/*
    /home/fredrik/code/repos/a/.venv/lib/python3.9/site-packages/b.pth
Proceed (y/n)? y
  Successfully uninstalled b-0.0.5

$ pip uninstall b
Found existing installation: b 0.0.4
Uninstalling b-0.0.4:
  Would remove:
    /home/fredrik/code/repos/a/.venv/lib/python3.9/site-packages/b-0.0.4.dist-info/*
Proceed (y/n)? y
  Successfully uninstalled b-0.0.4

$ pip uninstall b
WARNING: Skipping b as it is not installed.

@ns-admetrics
Copy link

ns-admetrics commented Apr 7, 2021

It seems however, that you will get a certain error message whenever pip has removed all the different versions:

Yup but parsing messages is silly :) Re. pip (off-topic...), if the semantics of uninstall are declarative then it should uninstall all versions if multiple, if the semantics are imperative then it should fail in this WARNING case.

@jkbecker
Copy link

jkbecker commented Apr 7, 2021

It's kind of annoying that pip doesn't exit with something non-zero when a package was not installed. I understand why it doesn't but if it did, one could just run uninstall in a while loop until the command fails to be sure it was completely cleaned up...

@sinoroc
Copy link

sinoroc commented Apr 7, 2021

pip show ProjectName seems to give a usable return value.

So, it could be possible to run pip uninstall ProjectName until pip show ProjectName returns a non-zero value, if that helps...

@jkbecker
Copy link

jkbecker commented Apr 7, 2021

Nice! Thank you, that is actually really helpful in my project as workaround for now.

@abn
Copy link
Member

abn commented Apr 9, 2021

Can you please try the fix at #3900.

Using pipx

pipx install --suffix=@3900 'poetry @ git+https://github.com/python-poetry/poetry.git@refs/pull/3900/head'
alias poetry=poetry@3900
poetry config virtualenvs.in-project true
poetry new example
pushd example
poetry install
find .venv/ -type d -name 'example*.dist-info'
poetry version minor
poetry install
find .venv/ -type d -name 'example*.dist-info'
poetry run pip show example
poetry version 0.1.0
poetry install
poetry run pip show example

Using a container (podman | docker)

podman run --rm -i --entrypoint bash python:3.8 <<EOF
set -xe
python -m pip install -q git+https://github.com/python-poetry/poetry.git@refs/pull/3900/head
poetry config virtualenvs.in-project true
poetry new example
pushd example
poetry install
find .venv/ -type d -name 'example*.dist-info'
poetry version minor
poetry install
find .venv/ -type d -name 'example*.dist-info'
poetry run pip show example
poetry version 0.1.0
poetry install
poetry run pip show example
EOF

@fredrikaverpil
Copy link
Contributor

fredrikaverpil commented Apr 9, 2021

@abn I just tried the #3900 fix (pipx approach) and I no longer see the lingering .dist-info folders accumulating. 👍
I tested this with path dependencies and git dependencies on Python 3.9.2 and 3.6.10.

However, I can see a couple of weird things. Maybe expected, but I figured I would mention them...

Output on Python 3.9.2 below. You can see how the "Installing..." never turns into the actual version on completed installation:

$ poetry@3900 install
Installing dependencies from lock file

Package operations: 6 installs, 0 updates, 0 removals

  • Installing certifi (2020.12.5): Installing...
  • Installing chardet (3.0.4): Installing...
  • Installing idna (2.8): Installing...
  • Installing urllib3 (1.22): Installing...
  • Installing requests (2.21.0): Installing...
  • Installing b (2.1.2 /home/fredrik/code/repos/b)

Installing the current project: a (1.0.0)

What was also a bit weird was that I somehow managed to break the output by first using a poetry 1.1.5:

$ poetry@work install
The virtual environment found in /home/fredrik/code/repos/a/.venv seems to be broken.
Recreating virtualenv a in /home/fredrik/code/repos/a/.venv
Installing dependencies from lock file

Package operations: 6 installs, 0 updates, 0 removals

  • Installing certifi (2020.12.5)
  • Installing chardet (3.0.4)
  • Installing idna (2.8)
  • Installing urllib3 (1.22)
  • Installing requests (2.21.0)
  • Installing b (2.1.2 /home/fredrik/code/repos/b)

Installing the current project: a (1.0.0)

and then I tried poetry@3900 again:

$ poetry@3900 install
Installing dependencies from lock file

Package operations: 0 installs, 5 updates, 0 removals

  • Updating certifi (2020.12.5 /home/fredrik/.cache/pyp
  • Updating certifi (2020.12.5 /home/fredrik/.cache/pypoetry/artifacts/d8/df/24/ed696681f34f8916b0aef99138db9a94e37d54684b9829af34a7fd4e39/certifi-2020.12.5-py2.py3-none-any.whl -> 2020.12.5): Installing...
  • Updating chardet (3.0.4 /home/fredrik/.cache/pypoetry/artifacts/8f/6f/1c/8085d730ad63c462222af30d0d01c4bd0caca5287e40b63c1fe8f529b7/chardet-3.0.4-py2.py3-none-any.whl -> 3.0.4): Installing...
  • Updating idna (2.8 /home/fredrik/.cache/pypoetry/artifacts/06/ea/a8/977db370296c645a9f7e1d772193cb28e1805639dace642b579a7d2cc0/idna-2.8-py2.py3-none-any.whl -> 2.8): Installing...
  • Updating urllib3 (1.22 /home/fredrik/.cache/pypoetry/artifacts/8a/d2/c3/8140b9cc76677757ce1397e48fec8a4c4c92e0ba44d0aa2aeb9bcf9b03/urllib3-1.22-py2.py3-none-any.whl -> 1.22): Installing...
  • Updating requests (2.21.0 /home/fredrik/.cache/pypoetry/artifacts/a2/2c/c6/5d7a41c3f8574633f2e590d06f41b47df7c5590c8ef73061d8941f8f45/requests-2.21.0-py2.py3-none-any.whl -> 2.21.0): Installing...

Installing the current project: a (1.0.0)

As you can see, that last time the output was very messy... I'm including a screenshot. Maybe this was because I was mixing poetry 1.1.5 and poetry@3900?

image

@abn
Copy link
Member

abn commented Apr 9, 2021

That is expected. Because of #3876. There are followup fixes for that in #3891 and #3900.

@abn
Copy link
Member

abn commented Apr 9, 2021

Oh, I misread the issue. Looks like there is a new issue in the lines not being updated during install. But this might not be related to the #3900 fix.

Also you will note that in 1.1.5 you will have seen;

The virtual environment found in /home/fredrik/code/repos/a/.venv seems to be broken.

This is because the new venv does not have pip in it (#2826).

@dstnceswmer
Copy link

Still seeing this behavior in v1.1.15

Copy link

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Mar 29, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
kind/bug Something isn't working as expected
Projects
None yet
8 participants