Latest poetry-core with poetry 1.0 caused pip error during tox building #3001

fredrikaverpil · 2020-09-30T06:31:21Z

I am on the latest Poetry version.
I have searched the issues of this repo and believe that this is not a duplicate.
If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

OS version and name: Linux
Poetry version: 1.0.10
Link of a Gist with the contents of your pyproject.toml file:

Issue

We use tox (with isolated_build=true) with poetry 1.0.10 and the build system in pyproject.toml looked like this:

[build-system]
requires = ["poetry-core>=1.0.0a9"]
build-backend = "poetry.core.masonry.api"

Jenkins log, from executing the tox testenv:

+ tox -e mytestenv
.package create: REDACTED/.tox/.package
.package installdeps: poetry-core>=1.0.0a9
mytestenv create: REDACTED/.tox/mytestenv
mytestenv inst: REDACTED/.tox/.tmp/package/1/pkg1-0.0.0.tar.gz
ERROR: invocation failed (exit code 2), logfile: REDACTED/.tox/mytestenv/log/mytestenv-1.log
================================== log start ===================================
Looking in indexes: https://REDACTED/api/pypi/pypi-remote/simple, https://REDACTED/api/pypi/custom/simple
Processing ./.tox/.tmp/package/1/pkg1-0.0.0.tar.gz
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
ERROR: Exception:
Traceback (most recent call last):
  File "REDACTED/.tox/mytestenv/lib/python3.7/site-packages/pip/_internal/cli/base_command.py", line 216, in _main
    status = self.run(options, args)
  File "REDACTED/.tox/mytestenv/lib/python3.7/site-packages/pip/_internal/cli/req_command.py", line 182, in wrapper
    return func(self, options, args)
  File "REDACTED/.tox/mytestenv/lib/python3.7/site-packages/pip/_internal/commands/install.py", line 325, in run
    reqs, check_supported_wheels=not options.target_dir
  File "REDACTED/.tox/mytestenv/lib/python3.7/site-packages/pip/_internal/resolution/legacy/resolver.py", line 183, in resolve
    discovered_reqs.extend(self._resolve_one(requirement_set, req))
  File "REDACTED/.tox/mytestenv/lib/python3.7/site-packages/pip/_internal/resolution/legacy/resolver.py", line 388, in _resolve_one
    abstract_dist = self._get_abstract_dist_for(req_to_install)
  File "REDACTED/.tox/mytestenv/lib/python3.7/site-packages/pip/_internal/resolution/legacy/resolver.py", line 340, in _get_abstract_dist_for
    abstract_dist = self.preparer.prepare_linked_requirement(req)
  File "REDACTED/.tox/mytestenv/lib/python3.7/site-packages/pip/_internal/operations/prepare.py", line 483, in prepare_linked_requirement
    req, self.req_tracker, self.finder, self.build_isolation,
  File "REDACTED/.tox/mytestenv/lib/python3.7/site-packages/pip/_internal/operations/prepare.py", line 91, in _get_prepared_distribution
    abstract_dist.prepare_distribution_metadata(finder, build_isolation)
  File "REDACTED/.tox/mytestenv/lib/python3.7/site-packages/pip/_internal/distributions/sdist.py", line 38, in prepare_distribution_metadata
    self._setup_isolation(finder)
  File "REDACTED/.tox/mytestenv/lib/python3.7/site-packages/pip/_internal/distributions/sdist.py", line 96, in _setup_isolation
    reqs = backend.get_requires_for_build_wheel()
  File "REDACTED/.tox/mytestenv/lib/python3.7/site-packages/pip/_vendor/pep517/wrappers.py", line 161, in get_requires_for_build_wheel
    'config_settings': config_settings
  File "REDACTED/.tox/mytestenv/lib/python3.7/site-packages/pip/_vendor/pep517/wrappers.py", line 265, in _call_hook
    raise BackendUnavailable(data.get('traceback', ''))
pip._vendor.pep517.wrappers.BackendUnavailable: Traceback (most recent call last):
  File "REDACTED/.tox/mytestenv/lib/python3.7/site-packages/pip/_vendor/pep517/_in_process.py", line 86, in _build_backend
    obj = import_module(mod_path)
  File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/tmp/pip-build-env-47q6ojew/overlay/lib/python3.7/site-packages/poetry/core/masonry/__init__.py", line 10, in <module>
    from .builder import Builder
  File "/tmp/pip-build-env-47q6ojew/overlay/lib/python3.7/site-packages/poetry/core/masonry/builder.py", line 6, in <module>
    from .builders.sdist import SdistBuilder
  File "/tmp/pip-build-env-47q6ojew/overlay/lib/python3.7/site-packages/poetry/core/masonry/builders/__init__.py", line 2, in <module>
    from .wheel import WheelBuilder
  File "/tmp/pip-build-env-47q6ojew/overlay/lib/python3.7/site-packages/poetry/core/masonry/builders/wheel.py", line 18, in <module>
    from packaging.tags import sys_tags
  File "/tmp/pip-build-env-47q6ojew/overlay/lib/python3.7/site-packages/poetry/core/_vendor/packaging/tags.py", line 7, in <module>
    import distutils.util
  File "REDACTED/.tox/mytestenv/lib/python3.7/site-packages/_distutils_hack/__init__.py", line 83, in create_module
    return importlib.import_module('setuptools._distutils')
  File "/usr/local/lib/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
ModuleNotFoundError: No module named 'setuptools'

Changing our build system to the following solved the problem:

[build-system]
requires = ["poetry==1.0.10"]
build-backend = "poetry.masonry.api"

The text was updated successfully, but these errors were encountered:

abn · 2020-09-30T10:44:57Z

@fredrikaverpil I believe the issue is the system default pip version. There was a bug in pip (circa 19.1) that meant that the PEP 517 build environment was not correctly created with the right versions of setuptools etc. With a poetry managed virtual environment this should not be an issue anymore as we will enforce the version of pip used via virtualenv (#2666).

Maybe #2826 will also remove this issue for externally (tox) managed virtual environments as well. In the interim, I would suggest that you get tox to upgrade pip used int he environment. Note that by default (if not using virtualenv's seed wheels), the system Python version's default pip version will be used, not the version you have in your active environment (ie. if you did pip install --upgrade --user pip (user is redundant in new versions of pip). The system default version ususaly is older on

abn · 2020-09-30T10:50:11Z

Also, this should not work unless your environment somehow already has poetry along with poetry-core

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.masonry.api"

fredrikaverpil · 2020-09-30T16:40:20Z

Oops, I meant to write:

Changing our build system to the following solved the problem:

[build-system]
requires = ["poetry==1.0.10"]
build-backend = "poetry.masonry.api"

fredrikaverpil · 2020-09-30T16:43:01Z

The funny thing is this happened over night without a change in tox.ini or pyproject.toml.

Ok, I will see which version of pip is in the system-Python.

finswimmer · 2020-10-01T19:11:53Z

The funny thing is this happened over night without a change in tox.ini or pyproject.toml.

The funny thing about that, is that a colleague of mine showed me this today with exact the same words 😄

We simply update poetry to the freshly released 1.1 and the problem vanished.

Shadowsong27 · 2020-10-02T03:13:37Z

I am seeing this as well (not in tox, but standard poetry install), less than 1 day since the last success built, strange thing is: we have always fixed the poetry version to 1.0.5, but once updated to the latest 1.1.0, it works fine.

nbraem · 2020-10-02T09:19:44Z

seeing this as well, workaround is to add setuptools to requires:

[build-system]
requires = ["poetry", "setuptools"]
build-backend = "poetry.masonry.api"

This way, you can still use poetry 1.1.0

abn · 2020-10-03T22:39:45Z

@fredrikaverpil I am unable to reproduce this in a clean environment,

$ podman run --rm -i --entrypoint bash python:3.7 <<EOF
> pip install --quiet --upgrade pip
> pip install --quiet poetry tox
> pushd /opt
> poetry new foobar
> pushd foobar
> cat > tox.ini <<TOXEOF
> [tox]
> minversion = 3.3.0
> isolated_build = True
> envlist = py37
> 
> [testenv]
> commands =
>     pip list
> TOXEOF
> cat pyproject.toml
> tox -e py37
> EOF
/opt /
Created package foobar in foobar
/opt/foobar /opt /
[tool.poetry]
name = "foobar"
version = "0.1.0"
description = ""
authors = ["Your Name <[email protected]>"]

[tool.poetry.dependencies]
python = "^3.7"

[tool.poetry.dev-dependencies]
pytest = "^5.2"

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"
.package create: /opt/foobar/.tox/.package
.package installdeps: poetry-core>=1.0.0
py37 create: /opt/foobar/.tox/py37
py37 inst: /opt/foobar/.tox/.tmp/package/1/foobar-0.1.0.tar.gz
py37 installed: foobar @ file:///opt/foobar/.tox/.tmp/package/1/foobar-0.1.0.tar.gz
py37 run-test-pre: PYTHONHASHSEED='2706476223'
py37 run-test: commands[0] | pip list
Package    Version
---------- -------
foobar     0.1.0
pip        20.2.3
setuptools 50.3.0
wheel      0.35.1
___________________________________ summary ____________________________________
  py37: commands succeeded
  congratulations :)

abn · 2020-10-03T23:14:03Z

@fredrikaverpil these issues may be of relevance for you pypa/setuptools#2353 pypa/setuptools#2376 pypa/pip#6264.

fredrikaverpil · 2020-10-05T05:53:25Z

Hm, a lot of people complaining about setuptools 50.0, but it all worked itself out fine on my end by using the following:

[build-system]
requires = ["poetry==1.0.10"]
build-backend = "poetry.masonry.api"

Does the above not make use of setuptools?

mtkennerly · 2020-10-05T09:27:47Z

@abn Those tickets are from a month ago, but I saw this issue start happening specifically within the last week in a GitHub Actions workflow, without any code changes in between.

No issue (2020-09-27): https://github.com/mtkennerly/poetry-dynamic-versioning/actions/runs/275347882
Issue began (2020-10-03): https://github.com/mtkennerly/poetry-dynamic-versioning/actions/runs/287057357

It occurs only in certain combinations. Python (3.7 | 3.8) + Poetry (1.0.2 | 1.0.5) fail, whereas the same Poetry versions work with Python (3.5 | 3.6), and Poetry (1.0.10 | 1.1.0rc1) work with all four versions of Python tested. It doesn't matter whether Poetry is installed with pip or with get-poetry.py (called "script" in the workflow matrix).

jhrmnn · 2020-10-09T15:13:48Z

I'm seeing the missing Setuptools error in environments created with Conda.

I'm not entirely sure what's causing the error. It seems to be related either to setuptools or to poetry, or maybe both. Thus, this is more or less a brute force fix. I've found the it here: python-poetry/poetry#3001

Installing the local directory from other projects using `pip` fails: ```sh $ cd chws_tools $ pip install ../east_asian_spacing ... ModuleNotFoundError: No module named 'setuptools' ``` This change can work around the problem as described at: python-poetry/poetry#3001

neersighted · 2022-10-05T14:53:41Z

Doesn't reproduce with any modern versions -- please open a new issue if you think you're hitting this.

github-actions · 2024-03-01T07:04:20Z

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

fredrikaverpil added kind/bug Something isn't working as expected status/triage This issue needs to be triaged labels Sep 30, 2020

abn mentioned this issue Sep 30, 2020

Could not build wheels for XXX which use PEP 517 #1516

Closed

2 tasks

jhabriel mentioned this issue Nov 19, 2020

Fix bug in pip editable install mode pmgbergen/porepy#486

Merged

neersighted closed this as not planned Won't fix, can't repro, duplicate, stale Oct 5, 2022

neersighted added status/needs-reproduction Issue needs a minimal reproduction to be confirmed and removed status/triage This issue needs to be triaged labels Oct 5, 2022

github-actions bot locked as resolved and limited conversation to collaborators Mar 1, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Latest poetry-core with poetry 1.0 caused pip error during tox building #3001

Latest poetry-core with poetry 1.0 caused pip error during tox building #3001

fredrikaverpil commented Sep 30, 2020 •

edited

Loading

abn commented Sep 30, 2020

abn commented Sep 30, 2020

fredrikaverpil commented Sep 30, 2020

fredrikaverpil commented Sep 30, 2020

finswimmer commented Oct 1, 2020

Shadowsong27 commented Oct 2, 2020

nbraem commented Oct 2, 2020 •

edited

Loading

abn commented Oct 3, 2020

abn commented Oct 3, 2020

fredrikaverpil commented Oct 5, 2020

mtkennerly commented Oct 5, 2020

jhrmnn commented Oct 9, 2020

neersighted commented Oct 5, 2022

github-actions bot commented Mar 1, 2024

Latest poetry-core with poetry 1.0 caused pip error during tox building #3001

Latest poetry-core with poetry 1.0 caused pip error during tox building #3001

Comments

fredrikaverpil commented Sep 30, 2020 • edited Loading

Issue

abn commented Sep 30, 2020

abn commented Sep 30, 2020

fredrikaverpil commented Sep 30, 2020

fredrikaverpil commented Sep 30, 2020

finswimmer commented Oct 1, 2020

Shadowsong27 commented Oct 2, 2020

nbraem commented Oct 2, 2020 • edited Loading

abn commented Oct 3, 2020

abn commented Oct 3, 2020

fredrikaverpil commented Oct 5, 2020

mtkennerly commented Oct 5, 2020

jhrmnn commented Oct 9, 2020

neersighted commented Oct 5, 2022

github-actions bot commented Mar 1, 2024

fredrikaverpil commented Sep 30, 2020 •

edited

Loading

nbraem commented Oct 2, 2020 •

edited

Loading