Poetry fails when two different dependency types are given for the same dependency

[roiefclassiq]

• I am on the latest Poetry version.

• I have searched the issues of this repo and believe that this is not a duplicate.

• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: MAc OS Darwin Kernel Version 21.3.0

• Poetry version: Poetry 1.1.14

Issue

poetry update consistently crashes when I try running it with a pyproject.toml file that contains the following:

pyscf = [
	{ url = {my_url.pyscf-2.0.1-cp38-cp38-macosx_12_0_arm64.whl}, markers="platform_machine == 'arm64'"},
	{ version="^2.0.1", markers= "platform_machine != 'arm64'" }
]

I chose to configure poetry in this way because the package that I need installed (pyscf) hasn't officially been generated for MacOS with the M1 chip. This way I am able to have devices that can get the pyscf package from pypi doing so, and M1 macs (and any other arm64) get an instance of the package I compiled locally and that I am storing on a server. However when configured like this poetry crashes before attempting to download or install any package.

The error that I receive is:

  9  ~/venvs/venv1/lib/python3.8/site-packages/clikit/console_application.py:131 in run
      129│             parsed_args = resolved_command.args
      130│ 
    → 131│             status_code = command.handle(parsed_args, io)
      132│         except KeyboardInterrupt:
      133│             status_code = 1

  8  ~/venvs/venv1/lib/python3.8/site-packages/clikit/api/command/command.py:120 in handle
      118│     def handle(self, args, io):  # type: (Args, IO) -> int
      119│         try:
    → 120│             status_code = self._do_handle(args, io)
      121│         except KeyboardInterrupt:
      122│             if io.is_debug():

  7  ~/venvs/venv1/lib/python3.8/site-packages/clikit/api/command/command.py:171 in _do_handle
      169│         handler_method = self._config.handler_method
      170│ 
    → 171│         return getattr(handler, handler_method)(args, io, self)
      172│ 
      173│     def __repr__(self):  # type: () -> str

  6  ~/venvs/venv1/lib/python3.8/site-packages/cleo/commands/command.py:92 in wrap_handle
       90│         self._command = command
       91│ 
    →  92│         return self.handle()
       93│ 
       94│     def handle(self):  # type: () -> Optional[int]

  5  ~/venvs/venv1/lib/python3.8/site-packages/poetry/console/commands/update.py:47 in handle
      45│         self._installer.update(True)
      46│ 
    → 47│         return self._installer.run()
      48│ 

  4  ~/venvs/venv1/lib/python3.8/site-packages/poetry/installation/installer.py:103 in run
      101│         local_repo = Repository()
      102│ 
    → 103│         return self._do_install(local_repo)
      104│ 
      105│     def dry_run(self, dry_run=True):  # type: (bool) -> Installer

  3  ~/venvs/venv1/lib/python3.8/site-packages/poetry/installation/installer.py:235 in _do_install
      233│             )
      234│ 
    → 235│             ops = solver.solve(use_latest=self._whitelist)
      236│         else:
      237│             self._io.write_line("Installing dependencies from lock file")

  2  ~/venvs/venv1/lib/python3.8/site-packages/poetry/puzzle/solver.py:65 in solve
       63│         with self._provider.progress():
       64│             start = time.time()
    →  65│             packages, depths = self._solve(use_latest=use_latest)
       66│             end = time.time()
       67│ 

  1  ~/venvs/venv1/lib/python3.8/site-packages/poetry/puzzle/solver.py:239 in _solve
      237│             packages = result.packages
      238│         except OverrideNeeded as e:
    → 239│             return self.solve_in_compatibility_mode(e.overrides, use_latest=use_latest)
      240│         except SolveFailure as e:
      241│             raise SolverProblemError(e)

  ValueError

  Package('pyscf', '2.0.1') is not in list

  at ~/venvs/venv1/lib/python3.8/site-packages/poetry/puzzle/solver.py:214 in solve_in_compatibility_mode
      210│                     packages.append(package)
      211│                     depths.append(_depths[index])
      212│                     continue
      213│                 else:
    → 214│                     idx = packages.index(package)
      215│                     pkg = packages[idx]
      216│                     depths[idx] = max(depths[idx], _depths[index])
      217│ 
      218│                     for dep in package.requires:

After some debugging I believe that this occurs because of two different instances of the class Package that are supposed to represent pyscf and compared while running the program. This all happens in the function "solve_in_compatibility_mode" in solver.py. In my case the list "packages" contains an instance of package as follows:

Package('pyscf', '2.0.1', source_type='url', source_url='{my_url}/pyscf-2.0.1-cp38-cp38-macosx_12_0_arm64.whl')

and the "package" variable is:

Package('pyscf', '2.0.1')

In this case the if statement "package not in packages" will return False, because the comparison sees them as the same instance. However later (line 214 in solver.py) the list index function is called and it scans the list and compares the hash of each object in the list to the hash of the object who's index it is looking for. In this case a ValueError is raised because the seperate instances of Package are different and therefore have different hashes.

I believe that this is unintentional and can be fixed.
Thanks

        packages = []
        depths = []
        for override in overrides:
            self._provider.debug(
                "<comment>Retrying dependency resolution "
                "with the following overrides ({}).</comment>".format(override)
            )
            self._provider.set_overrides(override)
            _packages, _depths = self._solve(use_latest=use_latest)
            for index, package in enumerate(_packages):
                if package not in packages:
                    packages.append(package)
                    depths.append(_depths[index])
                    continue
                else:
                    **idx = packages.index(package)**
                    pkg = packages[idx]
                    depths[idx] = max(depths[idx], _depths[index])

[dimbleby]

Try 1.2.0rc1.

I don't know whether this does better or not because you haven't shared a repro that anyone else can run, but it's definitely worth a try.

If it doesn't do better, please provide a repro that others can work with.

[roiefclassiq]

Adding the following to your pyproject.toml file will cause the same behavior

pyscf = [
{ git = "https://github.com/pyscf/pyscf", tag = "v2.0.1", markers= "platform_machine == 'arm64'"},
{ version="^2.0.1", markers= "platform_machine != 'arm64'" }
]

[dimbleby]

and did you try with 1.2.0rc1?

[ralbertazzi]

Closing as stale and duplicate of #7722

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.