External tools and constraints generated by carat on beta versions (^nn.mmbn)

[jdinunzio]

• Poetry version: 1.2.1
• Python version: 3.10.7
• OS version and name: Ubuntu 20.04.5
• pyproject.toml: https://gist.github.com/jdinunzio/88eab585ffbc687717de91cf8c291a8c

• I am on the latest stable Poetry version, installed using a recommended method.
• I have searched the issues of this repo and believe that this is not a duplicate.
• I have consulted the FAQ and blog for any relevant entries or release notes.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option) and have included the output below.

Issue

Step to reproduce:

• Create a python distribution for a library, let's call it mylib.
• mylib depends on a package whose current version is in beta (or other suffix), e.g. opentelemetry-instrumentation-logging = "^0.34b0"
• Build your library: poetry install; poetry build. The library will be built successfully, and the dependency version will be constraint with 'opentelemetry-instrumentation-logging>=0.34b0,<0.34'.
• Try to install the library or to add it as a dependency to a different poetry project.

Expected behaviour:

The library will be successfully installed or added.

Actual Behaviour:

The library will fail to install, since it can't satisfy the constraints of the given dependency ('opentelemetry-instrumentation-logging>=0.34b0,<0.34' in the above example).

Comments

The root cause seems to be the version constraints imposed in beta packages, since pip install equally fails at installing a package greater than beta but less than the version:

> pip install "opentelemetry-instrumentation-logging>=0.34b0,<0.34"
ERROR: Could not find a version that satisfies the requirement opentelemetry-instrumentation-logging<0.34,>=0.34b0 (from versions: 0.19b0, 0.20b0, 0.21b0, 0.22b0, 0.23b0, 0.23b2, 0.24b0, 0.25b0, 0.25b1, 0.25b2, 0.26b1, 0.27b0, 0.28b0, 0.28b1, 0.29b0, 0.30b0, 0.30b1, 0.31b0, 0.32b0, 0.33b0, 0.34b0)
ERROR: No matching distribution found for opentelemetry-instrumentation-logging<0.34,>=0.34b0

Nevertheless, since poetry was able to install the dependency while building the library, and poetry converted the ^ dependency spec in a range, the expectation is that such range should be able to be fulfilled later.

Possible solutions:

• Version comparison logic makes nn.mmbn less than nn.mm.
• When adding dependencies with beta versions, poetry prompts a warning indicating that they probably should be pinned to exact versions and not greater than x.
• (Probably undesirable: beta specs ceiling includes non-beta releases).

Error log

When trying to add the library as a dependency in another poetry project:

(.venv) jose@CA-LAP-315~/p/t/p/myapp> poetry add -vvv ../mylib/dist/mylib-0.1.0-py3-none-any.whl
Loading configuration file /home/jose/.config/pypoetry/config.toml
Using virtualenv: /home/jose/prog/tmp/poetrybug/myapp/.venv
Project environment contains an empty path in sys_path, ignoring.

Updating dependencies
Resolving dependencies...
   1: fact: myapp is 0.1.0
   1: derived: myapp
   1: fact: myapp depends on mylib (0.1.0)
   1: selecting myapp (0.1.0)
   1: derived: mylib (0.1.0) @ ../mylib/dist/mylib-0.1.0-py3-none-any.whl
   1: fact: mylib (0.1.0) depends on opentelemetry-exporter-otlp (>=1.13.0,<2.0.0)
   1: fact: mylib (0.1.0) depends on opentelemetry-exporter-otlp-proto-http (>=1.13.0,<2.0.0)
   1: fact: mylib (0.1.0) depends on opentelemetry-instrumentation-logging (<empty>)
   1: fact: mylib (0.1.0) depends on opentelemetry-sdk (>=1.13.0,<2.0.0)
   1: selecting mylib (0.1.0 /home/jose/prog/tmp/poetrybug/mylib/dist/mylib-0.1.0-py3-none-any.whl)
   1: derived: opentelemetry-sdk (>=1.13.0,<2.0.0)
   1: derived: opentelemetry-instrumentation-logging (<empty>)
   1: derived: opentelemetry-exporter-otlp-proto-http (>=1.13.0,<2.0.0)
   1: derived: opentelemetry-exporter-otlp (>=1.13.0,<2.0.0)
[keyring.backend] Loading KWallet
[keyring.backend] Loading SecretService
[keyring.backend] Loading Windows
[keyring.backend] Loading chainer
[keyring.backend] Loading libsecret
[keyring.backend] Loading macOS
Creating new session for pypi.org
Source (PyPI): 1 packages found for opentelemetry-sdk >=1.13.0,<2.0.0
Source (PyPI): 0 packages found for opentelemetry-instrumentation-logging <empty>
Falling back to installed packages to discover metadata for opentelemetry-instrumentation-logging
Found 0 compatible packages for opentelemetry-instrumentation-logging
Source (PyPI): 1 packages found for opentelemetry-exporter-otlp-proto-http >=1.13.0,<2.0.0
Source (PyPI): 1 packages found for opentelemetry-exporter-otlp >=1.13.0,<2.0.0
   1: fact: no versions of opentelemetry-instrumentation-logging match <empty>
   1: conflict: no versions of opentelemetry-instrumentation-logging match <empty>
   1: derived: not opentelemetry-instrumentation-logging (<empty>)
   1: conflict: mylib (0.1.0) depends on opentelemetry-instrumentation-logging (<empty>)
   1: ! not opentelemetry-instrumentation-logging (<empty>) is satisfied by not opentelemetry-instrumentation-logging (<empty>)
   1: ! which is caused by "no versions of opentelemetry-instrumentation-logging match <empty>"
   1: ! thus: mylib is forbidden
   1: ! mylib (0.1.0) @ file:///home/jose/prog/tmp/poetrybug/mylib/dist/mylib-0.1.0-py3-none-any.whl is satisfied by mylib (0.1.0) @ ../mylib/dist/mylib-0.1.0-py3-none-any.whl
   1: ! which is caused by "myapp depends on mylib (0.1.0) @ ../mylib/dist/mylib-0.1.0-py3-none-any.whl"
   1: ! thus: version solving failed
   1: Version solving took 0.206 seconds.
   1: Tried 1 solutions.

  Stack trace:

  4  ~/.asdf/installs/python/3.10.7/lib/python3.10/site-packages/poetry/puzzle/solver.py:151 in _solve
      149│ 
      150│         try:
    → 151│             result = resolve_version(
      152│                 self._package, self._provider, locked=locked, use_latest=use_latest
      153│             )

  3  ~/.asdf/installs/python/3.10.7/lib/python3.10/site-packages/poetry/mixology/__init__.py:24 in resolve_version
       22│     solver = VersionSolver(root, provider, locked=locked, use_latest=use_latest)
       23│ 
    →  24│     return solver.solve()
       25│ 

  2  ~/.asdf/installs/python/3.10.7/lib/python3.10/site-packages/poetry/mixology/version_solver.py:126 in solve
      124│             next: str | None = self._root.name
      125│             while next is not None:
    → 126│                 self._propagate(next)
      127│                 next = self._choose_package_version()
      128│ 

  1  ~/.asdf/installs/python/3.10.7/lib/python3.10/site-packages/poetry/mixology/version_solver.py:165 in _propagate
      163│                     # where that incompatibility will allow us to derive new assignments
      164│                     # that avoid the conflict.
    → 165│                     root_cause = self._resolve_conflict(incompatibility)
      166│ 
      167│                     # Back jumping erases all the assignments we did at the previous

  SolveFailure

  Because mylib (0.1.0) @ file:///home/jose/prog/tmp/poetrybug/mylib/dist/mylib-0.1.0-py3-none-any.whl depends on opentelemetry-instrumentation-logging (<empty>) which doesn't match any versions, mylib is forbidden.
  So, because myapp depends on mylib (0.1.0) @ ../mylib/dist/mylib-0.1.0-py3-none-any.whl, version solving failed.

  at ~/.asdf/installs/python/3.10.7/lib/python3.10/site-packages/poetry/mixology/version_solver.py:364 in _resolve_conflict
      360│             )
      361│             self._log(f'! which is caused by "{most_recent_satisfier.cause}"')
      362│             self._log(f"! thus: {incompatibility}")
      363│ 
    → 364│         raise SolveFailure(incompatibility)
      365│ 
      366│     def _choose_package_version(self) -> str | None:
      367│         """
      368│         Tries to select a version of a required package.

The following error occurred when trying to handle this error:


  Stack trace:

  11  ~/.asdf/installs/python/3.10.7/lib/python3.10/site-packages/cleo/application.py:329 in run
       327│ 
       328│             try:
     → 329│                 exit_code = self._run(io)
       330│             except Exception as e:
       331│                 if not self._catch_exceptions:

  10  ~/.asdf/installs/python/3.10.7/lib/python3.10/site-packages/poetry/console/application.py:185 in _run
       183│         self._load_plugins(io)
       184│ 
     → 185│         exit_code: int = super()._run(io)
       186│         return exit_code
       187│ 

   9  ~/.asdf/installs/python/3.10.7/lib/python3.10/site-packages/cleo/application.py:423 in _run
       421│             io.input.set_stream(stream)
       422│ 
     → 423│         exit_code = self._run_command(command, io)
       424│         self._running_command = None
       425│ 

   8  ~/.asdf/installs/python/3.10.7/lib/python3.10/site-packages/cleo/application.py:465 in _run_command
       463│ 
       464│         if error is not None:
     → 465│             raise error
       466│ 
       467│         return event.exit_code

   7  ~/.asdf/installs/python/3.10.7/lib/python3.10/site-packages/cleo/application.py:449 in _run_command
       447│ 
       448│             if event.command_should_run():
     → 449│                 exit_code = command.run(io)
       450│             else:
       451│                 exit_code = ConsoleCommandEvent.RETURN_CODE_DISABLED

   6  ~/.asdf/installs/python/3.10.7/lib/python3.10/site-packages/cleo/commands/base_command.py:119 in run
       117│         io.input.validate()
       118│ 
     → 119│         status_code = self.execute(io)
       120│ 
       121│         if status_code is None:

   5  ~/.asdf/installs/python/3.10.7/lib/python3.10/site-packages/cleo/commands/command.py:83 in execute
        81│ 
        82│         try:
     →  83│             return self.handle()
        84│         except KeyboardInterrupt:
        85│             return 1

   4  ~/.asdf/installs/python/3.10.7/lib/python3.10/site-packages/poetry/console/commands/add.py:255 in handle
       253│         self.installer.whitelist([r["name"] for r in requirements])
       254│ 
     → 255│         status = self.installer.run()
       256│ 
       257│         if status == 0 and not self.option("dry-run"):

   3  ~/.asdf/installs/python/3.10.7/lib/python3.10/site-packages/poetry/installation/installer.py:114 in run
       112│             self._execute_operations = False
       113│ 
     → 114│         return self._do_install()
       115│ 
       116│     def dry_run(self, dry_run: bool = True) -> Installer:

   2  ~/.asdf/installs/python/3.10.7/lib/python3.10/site-packages/poetry/installation/installer.py:247 in _do_install
       245│                 source_root=self._env.path.joinpath("src")
       246│             ):
     → 247│                 ops = solver.solve(use_latest=self._whitelist).calculate_operations()
       248│         else:
       249│             self._io.write_line("Installing dependencies from lock file")

   1  ~/.asdf/installs/python/3.10.7/lib/python3.10/site-packages/poetry/puzzle/solver.py:73 in solve
        71│         with self._provider.progress():
        72│             start = time.time()
     →  73│             packages, depths = self._solve(use_latest=use_latest)
        74│             end = time.time()
        75│ 

  SolverProblemError

  Because mylib (0.1.0) @ file:///home/jose/prog/tmp/poetrybug/mylib/dist/mylib-0.1.0-py3-none-any.whl depends on opentelemetry-instrumentation-logging (<empty>) which doesn't match any versions, mylib is forbidden.
  So, because myapp depends on mylib (0.1.0) @ ../mylib/dist/mylib-0.1.0-py3-none-any.whl, version solving failed.

  at ~/.asdf/installs/python/3.10.7/lib/python3.10/site-packages/poetry/puzzle/solver.py:159 in _solve
      155│             packages = result.packages
      156│         except OverrideNeeded as e:
      157│             return self.solve_in_compatibility_mode(e.overrides, use_latest=use_latest)
      158│         except SolveFailure as e:
    → 159│             raise SolverProblemError(e)
      160│ 
      161│         combined_nodes = depth_first_search(PackageNode(self._package, packages))
      162│         results = dict(aggregate_package_nodes(nodes) for nodes in combined_nodes)
      163│ 

[neersighted]

cc @dimbleby @radoering -- I lack the expertise to tell if Poetry is generating reasonable constraints here (or if we're allowing something through constraints we should not)

[dimbleby]

The root cause seems to be the version constraints imposed in beta packages

duplicate #6519, fixed at python-poetry/poetry-core#475

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.