poetry 1.2.0b3 lock fails with no output

[vstrimaitis]

• I am on the latest Poetry version.
• I have searched the issues of this repo and believe that this is not a duplicate.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: macOS Monterey 12.4
• Poetry version: 1.2.0b3
• Link of a Gist with the contents of your pyproject.toml file: click

Issue

I've discovered a case where the latest pre-release version of Poetry (1.2.0b3) fails to lock dependencies in some cases. Unfortunately, I haven't been able to track down why exactly this happens, but I do have a concrete example (linked above).

Steps to reproduce:

• Install Poetry 1.2.0b3 (I installed it with pipx install --suffix @1.2.0b3 'poetry==1.2.0b3' to distinguish from my other installations)
• Create a pyproject.toml with kfp = "1.8.12" and gsutil = "^5.6.0"
• Run [email protected] lock

The output I get is:

$ [email protected] lock
Updating dependencies
Resolving dependencies... (0.1s)



$

Running the same command with -vvv:

$ [email protected] lock -vvv
Loading configuration file /Users/vstrimaitis/Library/Preferences/pypoetry/config.toml
Using virtualenv: /Users/vstrimaitis/dev/random/reproduce-poetry-lock-bug/.venv
Project environment contains an empty path in sys_path, ignoring.
Updating dependencies
Resolving dependencies...
   1: fact: reproduce-poetry-lock-bug is 0.1.0
   1: derived: reproduce-poetry-lock-bug
   1: fact: reproduce-poetry-lock-bug depends on kfp (1.8.12)
   1: fact: reproduce-poetry-lock-bug depends on gsutil (^5.6.0)
   1: selecting reproduce-poetry-lock-bug (0.1.0)
   1: derived: gsutil (>=5.6.0,<6.0.0)
   1: derived: kfp (==1.8.12)
[keyring.backend] Loading KWallet
[keyring.backend] Loading SecretService
[keyring.backend] Loading Windows
[keyring.backend] Loading chainer
[keyring.backend] Loading libsecret
[keyring.backend] Loading macOS
Creating new session for pypi.org
Source (PyPI): No release information found for gsutil-3.33, skipping
Source (PyPI): 6 packages found for gsutil >=5.6.0,<6.0.0
Source (PyPI): 1 packages found for kfp 1.8.12
   1: fact: kfp (1.8.12) depends on absl-py (>=0.9,<2)
   1: fact: kfp (1.8.12) depends on PyYAML (>=5.3,<6)
   1: fact: kfp (1.8.12) depends on google-api-core (>=1.31.5,<2.0.0 || >2.3.0,<3.0.0dev)
   1: fact: kfp (1.8.12) depends on google-cloud-storage (>=1.20.0,<2)
   1: fact: kfp (1.8.12) depends on kubernetes (>=8.0.0,<19)
   1: fact: kfp (1.8.12) depends on google-api-python-client (>=1.7.8,<2)
   1: fact: kfp (1.8.12) depends on google-auth (>=1.6.1,<2)
   1: fact: kfp (1.8.12) depends on requests-toolbelt (>=0.8.0,<1)
   1: fact: kfp (1.8.12) depends on cloudpickle (>=2.0.0,<3)
   1: fact: kfp (1.8.12) depends on kfp-server-api (>=1.1.2,<2.0.0)
   1: fact: kfp (1.8.12) depends on jsonschema (>=3.0.1,<4)
   1: fact: kfp (1.8.12) depends on tabulate (>=0.8.6,<1)
   1: fact: kfp (1.8.12) depends on click (>=7.1.2,<9)
   1: fact: kfp (1.8.12) depends on Deprecated (>=1.2.7,<2)
   1: fact: kfp (1.8.12) depends on strip-hints (>=0.1.8,<1)
   1: fact: kfp (1.8.12) depends on docstring-parser (>=0.7.3,<1)
   1: fact: kfp (1.8.12) depends on kfp-pipeline-spec (>=0.1.14,<0.2.0)
   1: fact: kfp (1.8.12) depends on fire (>=0.3.1,<1)
   1: fact: kfp (1.8.12) depends on protobuf (>=3.13.0,<4)
   1: fact: kfp (1.8.12) depends on uritemplate (>=3.0.1,<4)
   1: fact: kfp (1.8.12) depends on pydantic (>=1.8.2,<2)
   1: fact: kfp (1.8.12) depends on typer (>=0.3.2,<1.0)
   1: selecting kfp (1.8.12)
   1: derived: typer (>=0.3.2,<1.0)
   1: derived: pydantic (>=1.8.2,<2)
   1: derived: uritemplate (>=3.0.1,<4)
   1: derived: protobuf (>=3.13.0,<4)
   1: derived: fire (>=0.3.1,<1)
   1: derived: kfp-pipeline-spec (>=0.1.14,<0.2.0)
   1: derived: docstring-parser (>=0.7.3,<1)
   1: derived: strip-hints (>=0.1.8,<1)
   1: derived: Deprecated (>=1.2.7,<2)
   1: derived: click (>=7.1.2,<9)
   1: derived: tabulate (>=0.8.6,<1)
   1: derived: jsonschema (>=3.0.1,<4)
   1: derived: kfp-server-api (>=1.1.2,<2.0.0)
   1: derived: cloudpickle (>=2.0.0,<3)
   1: derived: requests-toolbelt (>=0.8.0,<1)
   1: derived: google-auth (>=1.6.1,<2)
   1: derived: google-api-python-client (>=1.7.8,<2)
   1: derived: kubernetes (>=8.0.0,<19)
   1: derived: google-cloud-storage (>=1.20.0,<2)
   1: derived: google-api-core (>=1.31.5,<2.0.0 || >2.3.0,<3.0.0dev)
   1: derived: PyYAML (>=5.3,<6)
   1: derived: absl-py (>=0.9,<2)
Source (PyPI): 7 packages found for typer >=0.3.2,<1.0
Source (PyPI): 3 packages found for pydantic >=1.8.2,<2
Source (PyPI): 1 packages found for uritemplate >=3.0.1,<4
Source (PyPI): 25 packages found for protobuf >=3.13.0,<4
Source (PyPI): 2 packages found for fire >=0.3.1,<1
Source (PyPI): 3 packages found for kfp-pipeline-spec >=0.1.14,<0.2.0
Source (PyPI): 11 packages found for docstring-parser >=0.7.3,<1
Source (PyPI): 3 packages found for strip-hints >=0.1.8,<1
Source (PyPI): 7 packages found for deprecated >=1.2.7,<2
Source (PyPI): 10 packages found for click >=7.1.2,<9
Source (PyPI): 5 packages found for tabulate >=0.8.6,<1
Source (PyPI): 5 packages found for jsonschema >=3.0.1,<4
Source (PyPI): 12 packages found for kfp-server-api >=1.1.2,<2.0.0
Source (PyPI): No release information found for cloudpickle-0.2.0, skipping
Source (PyPI): 2 packages found for cloudpickle >=2.0.0,<3
Source (PyPI): No release information found for requests-toolbelt-0.0.0, skipping
Source (PyPI): 3 packages found for requests-toolbelt >=0.8.0,<1
Source (PyPI): 62 packages found for google-auth >=1.6.1,<2
Source (PyPI): Unable to parse version "1.0beta5prerelease" for the google-api-python-client package, skipping
Source (PyPI): Unable to parse version "1.0beta5prerelease2" for the google-api-python-client package, skipping
Source (PyPI): 28 packages found for google-api-python-client >=1.7.8,<2
Source (PyPI): 13 packages found for kubernetes >=8.0.0,<19
Source (PyPI): No release information found for google-cloud-storage-0.0.1, skipping
Source (PyPI): 37 packages found for google-cloud-storage >=1.20.0,<2
Source (PyPI): 16 packages found for google-api-core >=1.31.5,<2.0.0 || >2.3.0,<3.0.0dev
Source (PyPI): No release information found for pyyaml-3.01, skipping
Source (PyPI): No release information found for pyyaml-3.02, skipping
Source (PyPI): No release information found for pyyaml-3.03, skipping
Source (PyPI): No release information found for pyyaml-3.04, skipping
Source (PyPI): No release information found for pyyaml-3.05, skipping
Source (PyPI): No release information found for pyyaml-3.06, skipping
Source (PyPI): No release information found for pyyaml-3.07, skipping
Source (PyPI): No release information found for pyyaml-3.08, skipping
Source (PyPI): No release information found for pyyaml-3.09, skipping
Source (PyPI): 4 packages found for pyyaml >=5.3,<6
Source (PyPI): No release information found for absl-py-0.0, skipping
Source (PyPI): 11 packages found for absl-py >=0.9,<2
   1: selecting uritemplate (3.0.1)
   1: fact: fire (0.4.0) depends on six (*)
   1: fact: fire (0.4.0) depends on termcolor (*)
   1: selecting fire (0.4.0)
   1: derived: termcolor
   1: derived: six
Source (PyPI): 7 packages found for termcolor *
Source (PyPI): 27 packages found for six *
   1: selecting cloudpickle (2.1.0)
   1: fact: pydantic (1.9.1) depends on typing-extensions (>=3.7.4.3)
   1: selecting pydantic (1.9.1)
   1: derived: typing-extensions (>=3.7.4.3)
Source (PyPI): 10 packages found for typing-extensions >=3.7.4.3
   1: fact: kfp-pipeline-spec (0.1.16) depends on protobuf (>=3.13.0,<4)
   1: selecting kfp-pipeline-spec (0.1.16)
   1: fact: strip-hints (0.1.10) depends on wheel (*)
   1: selecting strip-hints (0.1.10)
   1: derived: wheel
Source (PyPI): 62 packages found for wheel *
   1: fact: requests-toolbelt (0.9.1) depends on requests (>=2.0.1,<3.0.0)
   1: selecting requests-toolbelt (0.9.1)
   1: derived: requests (>=2.0.1,<3.0.0)
Source (PyPI): No release information found for requests-0.0.1, skipping
Source (PyPI): No release information found for requests-0.12.01, skipping
Source (PyPI): No release information found for requests-2.15.0, skipping
Source (PyPI): 66 packages found for requests >=2.0.1,<3.0.0
   1: selecting pyyaml (5.4.1)
   1: selecting tabulate (0.8.10)
   1: fact: jsonschema (3.2.0) depends on attrs (>=17.4.0)
   1: fact: jsonschema (3.2.0) depends on pyrsistent (>=0.14.0)
   1: fact: jsonschema (3.2.0) depends on setuptools (*)
   1: fact: jsonschema (3.2.0) depends on six (>=1.11.0)
   1: selecting jsonschema (3.2.0)
   1: derived: six (>=1.11.0)
   1: derived: setuptools
   1: derived: pyrsistent (>=0.14.0)
   1: derived: attrs (>=17.4.0)
Source (PyPI): No release information found for setuptools-13.0, skipping
Source (PyPI): 478 packages found for setuptools *
Source (PyPI): 28 packages found for pyrsistent >=0.14.0
Source (PyPI): 13 packages found for attrs >=17.4.0
   1: fact: gsutil (5.11) depends on argcomplete (>=1.9.4)
   1: fact: gsutil (5.11) depends on crcmod (>=1.7)
   1: fact: gsutil (5.11) depends on fasteners (>=0.14.1)
   1: fact: gsutil (5.11) depends on gcs-oauth2-boto-plugin (>=3.0)
   1: fact: gsutil (5.11) depends on google-apitools (>=0.5.32)
   1: fact: gsutil (5.11) depends on httplib2 (>=0.20.4)
   1: fact: gsutil (5.11) depends on google-reauth (>=0.1.0)
   1: fact: gsutil (5.11) depends on monotonic (>=1.4)
   1: fact: gsutil (5.11) depends on pyOpenSSL (>=0.13)
   1: fact: gsutil (5.11) depends on retry_decorator (>=1.0.0)
   1: fact: gsutil (5.11) depends on six (>=1.12.0)
   1: fact: gsutil (5.11) depends on google-auth (>=2.5.0)
   1: selecting gsutil (5.11)
   1: derived: google-auth[aiohttp] (>=2.5.0)
   1: Version solving took 0.153 seconds.
   1: Tried 1 solutions.

  Stack trace:

  18  ~/.local/pipx/venvs/poetry@1-2-0b3/lib/python3.10/site-packages/cleo/application.py:329 in run
       327│
       328│             try:
     → 329│                 exit_code = self._run(io)
       330│             except Exception as e:
       331│                 if not self._catch_exceptions:

  17  ~/.local/pipx/venvs/poetry@1-2-0b3/lib/python3.10/site-packages/poetry/console/application.py:188 in _run
       186│         self._load_plugins(io)
       187│
     → 188│         exit_code: int = super()._run(io)
       189│         return exit_code
       190│

  16  ~/.local/pipx/venvs/poetry@1-2-0b3/lib/python3.10/site-packages/cleo/application.py:423 in _run
       421│             io.input.set_stream(stream)
       422│
     → 423│         exit_code = self._run_command(command, io)
       424│         self._running_command = None
       425│

  15  ~/.local/pipx/venvs/poetry@1-2-0b3/lib/python3.10/site-packages/cleo/application.py:465 in _run_command
       463│
       464│         if error is not None:
     → 465│             raise error
       466│
       467│         return event.exit_code

  14  ~/.local/pipx/venvs/poetry@1-2-0b3/lib/python3.10/site-packages/cleo/application.py:449 in _run_command
       447│
       448│             if event.command_should_run():
     → 449│                 exit_code = command.run(io)
       450│             else:
       451│                 exit_code = ConsoleCommandEvent.RETURN_CODE_DISABLED

  13  ~/.local/pipx/venvs/poetry@1-2-0b3/lib/python3.10/site-packages/cleo/commands/base_command.py:119 in run
       117│         io.input.validate()
       118│
     → 119│         status_code = self.execute(io)
       120│
       121│         if status_code is None:

  12  ~/.local/pipx/venvs/poetry@1-2-0b3/lib/python3.10/site-packages/cleo/commands/command.py:83 in execute
        81│
        82│         try:
     →  83│             return self.handle()
        84│         except KeyboardInterrupt:
        85│             return 1

  11  ~/.local/pipx/venvs/poetry@1-2-0b3/lib/python3.10/site-packages/poetry/console/commands/lock.py:54 in handle
        52│         self._installer.lock(update=not self.option("no-update"))
        53│
     →  54│         return self._installer.run()
        55│

  10  ~/.local/pipx/venvs/poetry@1-2-0b3/lib/python3.10/site-packages/poetry/installation/installer.py:111 in run
       109│             self._execute_operations = False
       110│
     → 111│         return self._do_install()
       112│
       113│     def dry_run(self, dry_run: bool = True) -> Installer:

   9  ~/.local/pipx/venvs/poetry@1-2-0b3/lib/python3.10/site-packages/poetry/installation/installer.py:244 in _do_install
       242│                 source_root=self._env.path.joinpath("src")
       243│             ):
     → 244│                 ops = solver.solve(use_latest=self._whitelist).calculate_operations()
       245│         else:
       246│             self._io.write_line("Installing dependencies from lock file")

   8  ~/.local/pipx/venvs/poetry@1-2-0b3/lib/python3.10/site-packages/poetry/puzzle/solver.py:73 in solve
        71│         with self._provider.progress():
        72│             start = time.time()
     →  73│             packages, depths = self._solve(use_latest=use_latest)
        74│             end = time.time()
        75│

   7  ~/.local/pipx/venvs/poetry@1-2-0b3/lib/python3.10/site-packages/poetry/puzzle/solver.py:141 in _solve
       139│
       140│         try:
     → 141│             result = resolve_version(
       142│                 self._package, self._provider, locked=locked, use_latest=use_latest
       143│             )

   6  ~/.local/pipx/venvs/poetry@1-2-0b3/lib/python3.10/site-packages/poetry/mixology/__init__.py:24 in resolve_version
        22│     solver = VersionSolver(root, provider, locked=locked, use_latest=use_latest)
        23│
     →  24│     return solver.solve()
        25│

   5  ~/.local/pipx/venvs/poetry@1-2-0b3/lib/python3.10/site-packages/poetry/mixology/version_solver.py:124 in solve
       122│             next: str | None = self._root.name
       123│             while next is not None:
     → 124│                 self._propagate(next)
       125│                 next = self._choose_package_version()
       126│

   4  ~/.local/pipx/venvs/poetry@1-2-0b3/lib/python3.10/site-packages/poetry/mixology/version_solver.py:153 in _propagate
       151│                     continue
       152│
     → 153│                 result = self._propagate_incompatibility(incompatibility)
       154│
       155│                 if result is _conflict:

   3  ~/.local/pipx/venvs/poetry@1-2-0b3/lib/python3.10/site-packages/poetry/mixology/version_solver.py:222 in _propagate_incompatibility
       220│         self._log(f"derived: {adverb}{unsatisfied.dependency}")
       221│
     → 222│         self._solution.derive(
       223│             unsatisfied.dependency, not unsatisfied.is_positive(), incompatibility
       224│         )

   2  ~/.local/pipx/venvs/poetry@1-2-0b3/lib/python3.10/site-packages/poetry/mixology/partial_solution.py:100 in derive
        98│         Adds an assignment of package as a derivation.
        99│         """
     → 100│         self._assign(
       101│             Assignment.derivation(
       102│                 dependency,

   1  ~/.local/pipx/venvs/poetry@1-2-0b3/lib/python3.10/site-packages/poetry/mixology/partial_solution.py:115 in _assign
       113│         """
       114│         self._assignments.append(assignment)
     → 115│         self._register(assignment)
       116│
       117│     def backtrack(self, decision_level: int) -> None:

  AssertionError



  at ~/.local/pipx/venvs/poetry@1-2-0b3/lib/python3.10/site-packages/poetry/mixology/partial_solution.py:160 in _register
      156│                 )
      157│                 old_positive = Term(dep, is_positive=True)
      158│         if old_positive is not None:
      159│             value = old_positive.intersect(assignment)
    → 160│             assert value is not None
      161│             self._positive[name] = value
      162│
      163│             return
      164│

The line 1: selecting gsutil (5.11) seems very weird to me because gsutil==5.11 is incompatible with kfp==1.8.12 because the first one depends on google-auth >=2.5.0 and the second one - on google-auth >=1.6.1,<2

With some other combinations of dependencies I also ran into a similar issue where there is no output from running [email protected] lock, but [email protected] lock -vvv reveals an OverrideNeeded exception, which sounds similar to #5791 (and also prints the same line with pandas and numpy). If needed, I can try to find a combination of dependencies that reveals this exact exception, but I don't have it right now.

If I try to do the same with Poetry version 1.2.0b2, everything works fine and resolves gsutil to version 5.6 because that's the latest version compatible with the other library (kfp).

[dimbleby]

introduced at #5770: which is not to say that fix was necessarily wrong... @radoering

[dimbleby]

Tweaking the #5770 fix so that it only kicks in if a direct-origin-dependency is involved seems to limit it sufficiently that it un-breaks this new example.

But I'm not sure whether this is actually sensible or it is just putting one dodgy fix on top of another

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.