Pip-complie fail on Python 3.11.0-beta.1 (pip._internal.exceptions.InstallationSubprocessError)

[Cheukting]

Description

Only in Python 3.11.0-beta.1 (pass on Python 3.10 or below)

pip._internal.exceptions.InstallationSubprocessError is raise when running command pip-compile --quiet --upgrade --rebuild --output-file=requirements.txt requirements.in

ERROR:pip.subprocessor:[present-diagnostic] pip subprocess to install build dependencies exited with 1
Traceback (most recent call last):
  File "/home/runner/work/terminusdb-client-python/terminusdb-client-python/.tox/deps/bin/pip-compile", line 8, in <module>
    sys.exit(cli())
             ^^^^^
  File "/home/runner/work/terminusdb-client-python/terminusdb-client-python/.tox/deps/lib/python3.11/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/terminusdb-client-python/terminusdb-client-python/.tox/deps/lib/python3.11/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/home/runner/work/terminusdb-client-python/terminusdb-client-python/.tox/deps/lib/python3.11/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/terminusdb-client-python/terminusdb-client-python/.tox/deps/lib/python3.11/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/terminusdb-client-python/terminusdb-client-python/.tox/deps/lib/python3.11/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/terminusdb-client-python/terminusdb-client-python/.tox/deps/lib/python3.11/site-packages/piptools/scripts/compile.py", line 466, in cli
    results = resolver.resolve(max_rounds=max_rounds)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/terminusdb-client-python/terminusdb-client-python/.tox/deps/lib/python3.11/site-packages/piptools/resolver.py", line 175, in resolve
    has_changed, best_matches = self._resolve_one_round()
                                ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/terminusdb-client-python/terminusdb-client-python/.tox/deps/lib/python3.11/site-packages/piptools/resolver.py", line 319, in _resolve_one_round
    their_constraints.extend(self._iter_dependencies(best_match))
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/terminusdb-client-python/terminusdb-client-python/.tox/deps/lib/python3.11/site-packages/piptools/resolver.py", line 428, in _iter_dependencies
    dependencies = self.repository.get_dependencies(ireq)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/terminusdb-client-python/terminusdb-client-python/.tox/deps/lib/python3.11/site-packages/piptools/repositories/pypi.py", line 243, in get_dependencies
    self._dependencies_cache[ireq] = self.resolve_reqs(
                                     ^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/terminusdb-client-python/terminusdb-client-python/.tox/deps/lib/python3.11/site-packages/piptools/repositories/pypi.py", line 206, in resolve_reqs
    results = resolver._resolve_one(reqset, ireq)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/terminusdb-client-python/terminusdb-client-python/.tox/deps/lib/python3.11/site-packages/pip/_internal/resolution/legacy/resolver.py", line 379, in _resolve_one
    dist = self._get_dist_for(req_to_install)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/terminusdb-client-python/terminusdb-client-python/.tox/deps/lib/python3.11/site-packages/pip/_internal/resolution/legacy/resolver.py", line 332, in _get_dist_for
    dist = self.preparer.prepare_linked_requirement(req)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/terminusdb-client-python/terminusdb-client-python/.tox/deps/lib/python3.11/site-packages/pip/_internal/operations/prepare.py", line [48](https://github.com/terminusdb/terminusdb-client-python/runs/6350948840?check_suite_focus=true#step:4:48)7, in prepare_linked_requirement
    return self._prepare_linked_requirement(req, parallel_builds)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/terminusdb-client-python/terminusdb-client-python/.tox/deps/lib/python3.11/site-packages/pip/_internal/operations/prepare.py", line [55](https://github.com/terminusdb/terminusdb-client-python/runs/6350948840?check_suite_focus=true#step:4:55)6, in _prepare_linked_requirement
    dist = _get_prepared_distribution(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/terminusdb-client-python/terminusdb-client-python/.tox/deps/lib/python3.11/site-packages/pip/_internal/operations/prepare.py", line 58, in _get_prepared_distribution
    abstract_dist.prepare_distribution_metadata(finder, build_isolation)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/terminusdb-client-python/terminusdb-client-python/.tox/deps/lib/python3.11/site-packages/pip/_internal/distributions/sdist.py", line 35, in prepare_distribution_metadata
    self._prepare_build_backend(finder)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/terminusdb-client-python/terminusdb-client-python/.tox/deps/lib/python3.11/site-packages/pip/_internal/distributions/sdist.py", line [56](https://github.com/terminusdb/terminusdb-client-python/runs/6350948840?check_suite_focus=true#step:4:56), in _prepare_build_backend
    self.req.build_env.install_requirements(
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/terminusdb-client-python/terminusdb-client-python/.tox/deps/lib/python3.11/site-packages/pip/_internal/build_env.py", line 202, in install_requirements
    self._install_requirements(
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/runner/work/terminusdb-client-python/terminusdb-client-python/.tox/deps/lib/python3.11/site-packages/pip/_internal/build_env.py", line 2[60](https://github.com/terminusdb/terminusdb-client-python/runs/6350948840?check_suite_focus=true#step:4:60), in _install_requirements
    call_subprocess(
    ^^^^^^^^^^^^^^^^
  File "/home/runner/work/terminusdb-client-python/terminusdb-client-python/.tox/deps/lib/python3.11/site-packages/pip/_internal/utils/subprocess.py", line 224, in call_subprocess
    raise error
    ^^^^^^^^^^^
pip._internal.exceptions.InstallationSubprocessError: pip subprocess to install build dependencies exited with 1
ERROR: InvocationError for command /home/runner/work/terminusdb-client-python/terminusdb-client-python/.tox/deps/bin/pip-compile --quiet --upgrade --rebuild --output-file=requirements.txt requirements.in (exited with code 1)

See https://github.com/terminusdb/terminusdb-client-python/runs/6350948840?check_suite_focus=true#step:4:45 and terminusdb/terminusdb-client-python#306

Expected behavior

Pass like Python 3.10

pip version

22.0.4

Python version

3.11.0-beta.1

OS

Ubuntu 20.04.4 (on GH action)

How to Reproduce

pip-compile --quiet --upgrade --rebuild --output-file=requirements.txt requirements.in

Output

No response

Code of Conduct

• I agree to follow the PSF Code of Conduct.

[DiddiLeija]

A few notes here. This could belong to https://github.com/jazzband/pip-tools (which is independent to pip, but uses its internal API), or maybe it's a legitimate pip issue. Anyway, if it is a pip-tools-specific issue, we might need to move something to ensure compatibility (or maybe not?).

One more question: does the pip-compile command you mentioned fails on a non-CI machine with Python 3.11?

[Cheukting]

"One more question: does the pip-compile command you mentioned fails on a non-CI machine with Python 3.11?" Yes, I run the same on 3.7 to 3.10 they all pass, only the 3.11 failed.

[pradyunsg]

If you're able to reproduce this with pip on its own, please provide a reproducer. Otherwise, please file an issue against pip-tools, as suggested.

[DiddiLeija]

Otherwise, please file an issue against pip-tools, as suggested.

And if that's the case, I'd suggest you to experiment and see if this is an issue with a certain pip issue, because that can help the pip-tools developers to find out what's changed.

[Cheukting]

Ok, I will have a look there. Thanks.

[Cheukting]

Just to tie things up, I discover that it is caused by the build failure for pandas on Python 3.11 while running the command pip install pandas on Python 3.11.0a7

Collecting pandas
  Using cached pandas-1.4.2.tar.gz (4.9 MB)
  Installing build dependencies ... error
  error: subprocess-exited-with-error
  
  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 1
  ╰─> [1556 lines of output]

The problem would be from the C compiler. I will look into that direction

[pradyunsg]

Yea, I think this is definitely an issue with pip-compile, since it's not presenting pip's errors in a manner compatible with how pip presents them now (see #10421 for context). I'd recommend filing an issue with pip-compile about this.