Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support including unsafe packages in exported requirements files #5137

Closed
2 tasks done
shaib opened this issue Feb 1, 2022 · 3 comments
Closed
2 tasks done

Support including unsafe packages in exported requirements files #5137

shaib opened this issue Feb 1, 2022 · 3 comments
Labels
kind/feature Feature requests/implementations

Comments

@shaib
Copy link

shaib commented Feb 1, 2022

  • I have searched the issues of this repo and believe that this is not a duplicate.
  • I have searched the documentation and believe that my question is not covered.

Hello, and thanks for Poetry.

In relatively recent versions of pip, there is a reqirement that In --require-hashes mode, all requirements must have their versions pinned (quoting the error message). That is not exactly accurate -- it is only dependencies that really need to be installed, which must be pinned; but when they do need to be installed, this applies to "unsafe" packages as well.

Poetry, at least as of 1.1.12, has no option to include unsafe packages in exported requirements files. This has recently led to breakage in a project I'm working on (which, unfortunately, I cannot share). The project is a web application which manages its dependencies through Poetry, and exports requirements for deployment on a PaaS.

The issue was triggered by an upgrade of Celery to version 5.2.3. In essence, we ran into celery/celery#7070 (the issue names pip-tools, but later users comment they experienced the same with Poetry):

  • Celery introduced a dependency on specific, recent versions of setuptools, using a range of versions (setuptools>=59.1.1,<59.7.0);
  • The pre-installed version of setuptools is typically not included in the range, so it must be upgraded;
  • But setuptools is an unsafe package, so it is not included in the exported requirements,
  • and so we get to the point where pip takes a requirements file where all entries have hashes -- implying --require-hashes -- and still has to install an unpinned dependency.

To reproduce:

$ poetry new fail && cd fail && poetry add celery==5.2.3 && poetry export -f requirements.txt -o requirements.txt && poetry run pip install -r requirements.txt

Of note, pip-compile (from pip-tools) has an --allow-unsafe option for this, and they plan to make that behavior the default -- see https://pypi.org/project/pip-tools/ (look for "Deprecations").

I suspect Poetry should do something similar.

Issues #3075 and #3472 seem to be related.

(FWIW, we considered two suboptimal-result workarounds -- Pinning Celery at 5.2.2, and doing away with hashes completely; the workaround chosen was a suboptimal-process one: We used pip-compile to generate a requirements-file with hashes for a suitable version of setuptools, and appended it to the requirements file exported by Poetry).

@shaib shaib added kind/feature Feature requests/implementations status/triage This issue needs to be triaged labels Feb 1, 2022
@dimbleby
Copy link
Contributor

It seems that poetry currently considers that there are no unsafe packages:

UNSAFE_PACKAGES: set[str] = set()

which is a slightly odd way to do it, but probably resolves this

@abn
Copy link
Member

abn commented May 16, 2022

@dimbleby is correct. This was resolved in #2826.

@abn abn removed the status/triage This issue needs to be triaged label May 16, 2022
@abn abn closed this as completed May 16, 2022
Copy link

github-actions bot commented Mar 2, 2024

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Mar 2, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
kind/feature Feature requests/implementations
Projects
None yet
Development

No branches or pull requests

3 participants