Poetry fails to install numpy from git with Python 3.12.0b2

[refparo]

• Poetry version: Poetry (version 1.5.1)
• Python version: output of poetry debug info:

Poetry
Version: 1.5.1
Python:  3.11.3

Virtualenv
Python:         3.12.0
Implementation: CPython
Path:           /home/paro/.cache/pypoetry/virtualenvs/sandbox-_6wzS4w7-py3.12
Executable:     /home/paro/.cache/pypoetry/virtualenvs/sandbox-_6wzS4w7-py3.12/bin/python
Valid:          True

System
Platform:   linux
OS:         posix
Python:     3.12.0
Path:       /home/paro/.pyenv/versions/3.12.0b2
Executable: /home/paro/.pyenv/versions/3.12.0b2/bin/python3.12

• OS version and name: Arch Linux
• pyproject.toml:

[tool.poetry]
name = "sandbox"
version = "0.1.0"
description = ""
authors = ["Paro <[email protected]>"]
readme = "README.md"

[tool.poetry.dependencies]
python = "^3.12"
numpy = {git = "https://github.com/numpy/numpy.git", rev = "c0585616c9b563f28cb6121bc5c67575168d6a16"}


[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

• I am on the latest stable Poetry version, installed using a recommended method. I use a version of Poetry from my system package manager.
• I have searched the issues of this repo and believe that this is not a duplicate.
• I have consulted the FAQ and blog for any relevant entries or release notes.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option) and have included the output below.

Loading configuration file /home/paro/.config/pypoetry/config.toml
Trying to detect current active python executable as specified in the config.
Found: /home/paro/.pyenv/versions/3.12.0b2/bin/python
Using virtualenv: /home/paro/.cache/pypoetry/virtualenvs/sandbox-_6wzS4w7-py3.12
[keyring.backend] Loading KWallet
[keyring.backend] Loading SecretService
[keyring.backend] Loading Windows
[keyring.backend] Loading chainer
[keyring.backend] Loading libsecret
[keyring.backend] Loading macOS
No suitable keyring backend found
No suitable keyring backends were found
Keyring is not available, credentials will be stored and retrieved from configuration files as plaintext.
[urllib3.connectionpool] Starting new HTTPS connection (1): github.com:443
[urllib3.connectionpool] https://github.com:443 "GET /numpy/numpy.git/info/refs?service=git-upload-pack HTTP/1.1" 200 None
[urllib3.connectionpool] Starting new HTTPS connection (2): github.com:443
[urllib3.connectionpool] https://github.com:443 "POST /numpy/numpy.git/git-upload-pack HTTP/1.1" 200 None
Cloning https://github.com/numpy/numpy.git at 'HEAD' to /home/paro/.cache/pypoetry/virtualenvs/sandbox-_6wzS4w7-py3.12/src/numpy

Updating dependencies
Resolving dependencies...
   1: fact: sandbox is 0.1.0
   1: derived: sandbox
   1: fact: sandbox depends on numpy (2.0.0.dev0)
   1: selecting sandbox (0.1.0)
   1: derived: numpy (2.0.0.dev0) @ git+https://github.com/numpy/numpy.git
   1: selecting numpy (2.0.0.dev0 c058561)
   1: Version solving took 0.003 seconds.
   1: Tried 1 solutions.

Finding the necessary packages for the current system

Package operations: 1 install, 0 updates, 0 removals

  • Installing numpy (2.0.0.dev0 c058561): Pending...
  • Installing numpy (2.0.0.dev0 c058561): Cloning...
  • Installing numpy (2.0.0.dev0 c058561): Preparing...
  • Installing numpy (2.0.0.dev0 c058561): Failed

  Stack trace:

  7  /usr/lib/python3.11/site-packages/poetry/installation/executor.py:286 in _execute_operation
      284│ 
      285│             try:
    → 286│                 result = self._do_execute_operation(operation)
      287│             except EnvCommandError as e:
      288│                 if e.e.returncode == -2:

  6  /usr/lib/python3.11/site-packages/poetry/installation/executor.py:388 in _do_execute_operation
      386│             return 0
      387│ 
    → 388│         result: int = getattr(self, f"_execute_{method}")(operation)
      389│ 
      390│         if result != 0:

  5  /usr/lib/python3.11/site-packages/poetry/installation/executor.py:508 in _execute_install
      506│ 
      507│     def _execute_install(self, operation: Install | Update) -> int:
    → 508│         status_code = self._install(operation)
      509│ 
      510│         self._save_url_reference(operation)

  4  /usr/lib/python3.11/site-packages/poetry/installation/executor.py:535 in _install
      533│         cleanup_archive: bool = False
      534│         if package.source_type == "git":
    → 535│             archive = self._prepare_git_archive(operation)
      536│             cleanup_archive = operation.package.develop
      537│         elif package.source_type == "file":

  3  /usr/lib/python3.11/site-packages/poetry/installation/executor.py:659 in _prepare_git_archive
      657│             )
      658│ 
    → 659│         archive = self._prepare_archive(operation, output_dir=output_dir)
      660│         if not package.develop:
      661│             package._source_url = original_url

  2  /usr/lib/python3.11/site-packages/poetry/installation/executor.py:612 in _prepare_archive
      610│         self._populate_hashes_dict(archive, package)
      611│ 
    → 612│         return self._chef.prepare(
      613│             archive, editable=package.develop, output_dir=output_dir
      614│         )

  1  /usr/lib/python3.11/site-packages/poetry/installation/chef.py:99 in prepare
       97│         if archive.is_dir():
       98│             destination = output_dir or Path(tempfile.mkdtemp(prefix="poetry-chef-"))
    →  99│             return self._prepare(archive, destination=destination, editable=editable)
      100│ 
      101│         return self._prepare_sdist(archive, destination=output_dir)

  ChefBuildError

  Backend subprocess exited when trying to invoke build_wheel
  
  + meson setup /home/paro/.cache/pypoetry/virtualenvs/sandbox-_6wzS4w7-py3.12/src/numpy /home/paro/.cache/pypoetry/virtualenvs/sandbox-_6wzS4w7-py3.12/src/numpy/.mesonpy-sx3idcve/build -Dbuildtype=release -Db_ndebug=if-release -Db_vscrt=md --native-file=/home/paro/.cache/pypoetry/virtualenvs/sandbox-_6wzS4w7-py3.12/src/numpy/.mesonpy-sx3idcve/build/meson-python-native-file.ini
  The Meson build system
  Version: 1.1.1
  Source dir: /home/paro/.cache/pypoetry/virtualenvs/sandbox-_6wzS4w7-py3.12/src/numpy
  Build dir: /home/paro/.cache/pypoetry/virtualenvs/sandbox-_6wzS4w7-py3.12/src/numpy/.mesonpy-sx3idcve/build
  Build type: native build
  Project name: NumPy
  Project version: 2.0.0.dev0
  C compiler for the host machine: cc (gcc 13.1.1 "cc (GCC) 13.1.1 20230429")
  C linker for the host machine: cc ld.bfd 2.40.0
  C++ compiler for the host machine: c++ (gcc 13.1.1 "c++ (GCC) 13.1.1 20230429")
  C++ linker for the host machine: c++ ld.bfd 2.40.0
  Cython compiler for the host machine: cython (cython 0.29.35)
  Host machine cpu family: x86_64
  Host machine cpu: x86_64
  
  ../../meson.build:42:22: ERROR: <PythonExternalProgram 'python' -> ['/tmp/tmpaq8hs74z/.venv/bin/python']> is not a valid python or it is missing distutils
  
  A full log can be found at /home/paro/.cache/pypoetry/virtualenvs/sandbox-_6wzS4w7-py3.12/src/numpy/.mesonpy-sx3idcve/build/meson-logs/meson-log.txt
  

  at /usr/lib/python3.11/site-packages/poetry/installation/chef.py:147 in _prepare
      143│ 
      144│                 error = ChefBuildError("\n\n".join(message_parts))
      145│ 
      146│             if error is not None:
    → 147│                 raise error from None
      148│ 
      149│             return path
      150│ 
      151│     def _prepare_sdist(self, archive: Path, destination: Path | None = None) -> Path:

Issue

With Python 3.12.0b2, Poetry fails to install numpy from git, while pip install installs the same revision normally.

Reproduction:

• poetry config virtualenvs.prefer-active-python true
• pyenv local 3.12.0b2
• poetry init
• poetry add git+https://github.com/numpy/numpy.git#c0585616c9b563f28cb6121bc5c67575168d6a16 -vvv
• Comparison: python -m pip install https://github.com/numpy/numpy.git#c0585616c9b563f28cb6121bc5c67575168d6a16 --verbose

[dimbleby]

as you can see, this error is coming from the meson subprocess. You should start there.

[darrylwest]

I'm having the same error with poetry 1.6.1 python3.12.0rc3 in a Debian 12 (bookworm) container.
• Installing numpy (1.25.2): Failed

ChefBuildError

Backend 'setuptools.build_meta:legacy' is not available.

at ~/.local/lib/python3.12/site-packages/poetry/installation/chef.py:147 in _prepare
143│
144│ error = ChefBuildError("\n\n".join(message_parts))
145│
146│ if error is not None:
→ 147│ raise error from None
148│
149│ return path
150│
151│ def _prepare_sdist(self, archive: Path, destination: Path | None = None) -> Path:

python -m pip install numpy works as expected.

[dimbleby]

that's not the same thing: but it's also not a poetry bug - you'll need to take this up with numpy too.

[netpoe]

I second this.
Using MacOS, Python 3.12.4:

  - Installing numpy (1.23.5): Failed

  ChefBuildError

  Backend 'setuptools.build_meta:__legacy__' is not available.
  
  Cannot import 'setuptools.build_meta'

  at ~/.local/pipx/venvs/poetry/lib/python3.12/site-packages/poetry/installation/chef.py:164 in _prepare
      160│ 
      161│                 error = ChefBuildError("\n\n".join(message_parts))
      162│ 
      163│             if error is not None:
    → 164│                 raise error from None
      165│ 
      166│             return path
      167│ 
      168│     def _prepare_sdist(self, archive: Path, destination: Path | None = None) -> Path:

Note: This error originates from the build backend, and is likely not a problem with poetry but with numpy (1.23.5) not supporting PEP 517 builds. You can verify this by running 'pip wheel --no-cache-dir --use-pep517 "numpy (==1.23.5)"'.

What should the workaround be?

[dimbleby]

Still not a poetry bug, this should be closed. Take bug reports and questions to numpy

[netpoe]

Still not a poetry bug, this should be closed. Take bug reports and questions to numpy

Indeed, not a poetry bug. It was a python version error. I was on 3.12+, and needed 3.10

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.