Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve detection of installed packages #1786

Merged
merged 1 commit into from
Jan 5, 2020

Conversation

sdispater
Copy link
Member

Pull Request Check List

  • Added tests for changed code.
  • Updated documentation for changed code.

In version 1.0.0, we changed the way the installed packages are detected to improve the detection of the type of packages (directory vs git dependencies for instance). However, this introduced issues where Poetry always reinstalled the packages because it was detecting the wrong type of package or because it was not retrieving it from the correct directory. This PR should fix most of these issues.

Fixes #1711
Fixes #1612

@sdispater sdispater requested a review from a team December 23, 2019 21:39
Comment on lines +775 to +778
if site_packages:
self._site_packages = site_packages[0]
else:
self._site_packages = dist_packages[0]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know where the site_packages property is used, so it isn't obvious to me, why only the first site_package is stored and why dist-packages are never stored when site_packages where found.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The site_packages property is used in the InstalledRepository class (see https://github.com/python-poetry/poetry/blob/master/poetry/repositories/installed_repository.py#L32).

We only need the "site-packages" directory where pip will install the dependencies which is, as far as I know, the site-packages in virtual environments and most OS and dist-packages for Debian-based distributions.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, ok I got it. I wonder if something with sysconfig would be more reliable? But as far as I can see, we must be able to detect if we are inside a virtual environment and of we are on a posix or nt system, to get the correct path (purelib seems to be the one we are interested in), are we?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for pointing me to sysconfig! Do you know if there are some systems where purelib and platlib are different? If there is none we could indeed use purelib and avoid the heuristics we are currently doing.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure. I've just discovered that module as well. There is something in PEP427 about the difference.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the end, it seems that all installed packages will reside in purelib and/or platlib, so to get populate the InstalledRepository we can just check both, I think. I'll test it out and see where it goes.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After testing it out it seems that using sysconfig won't help us. While it works in some cases, for Debian-based systems, for instance, it won't return the directory we need. We could use site.getsitepackages() for non-virtual environments but we would need to find a way to replicate it for virtual environments which would lead to using the same heuristic this PR already uses.

I think we can leave the PR as-is for now since I tested it on Ubuntu, MacOS and Windows without any issue and it fixes the issues I mentioned above.

Copy link
Member

@finswimmer finswimmer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then go ahead 😃

Copy link

github-actions bot commented Mar 1, 2024

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Mar 1, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

poetry re-installs up-to-date packages when virtualenvs.create is false All dependencies are always updated
2 participants