Inclusion of git dependency causes poetry to TypeError

[hlieberman]

• I am on the latest Poetry version.

• I have searched the issues of this repo and believe that this is not a duplicate.

• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: Debian bullseye/sid

• Poetry version: Poetry version 1.1.4

• Link of a Gist with the contents of your pyproject.toml file: https://gist.github.com/hlieberman/3057f3b572355cc6699cc44bdf352979

Issue

It's possible this is related to the now-closed issue #2027, but the solutions listed in that issue don't appear to be applicable in this situation (there are no .egg-info files involved).

After adding a git repository dependency to my project, all poetry commands involving dependency resolution now fail with a TypeError expected string or bytes-like object from poetry.helpers.canonicalize_name.

poetry show, poetry install, and poetry update all cause these errors. The project is essentially barren; there's no actual code put down yet, so it should be relatively simple to replicate.

Exception output:

Using virtualenv: /home/hlieberman/.cache/pypoetry/virtualenvs/renspell-yOBpFtFN-py3.8

  Stack trace:

  17  ~/.local/lib/python3.8/site-packages/clikit/console_application.py:131 in run
       129│             parsed_args = resolved_command.args
       130│
     → 131│             status_code = command.handle(parsed_args, io)
       132│         except KeyboardInterrupt:
       133│             status_code = 1

  16  ~/.local/lib/python3.8/site-packages/clikit/api/command/command.py:120 in handle
       118│     def handle(self, args, io):  # type: (Args, IO) -> int
       119│         try:
     → 120│             status_code = self._do_handle(args, io)
       121│         except KeyboardInterrupt:
       122│             if io.is_debug():

  15  ~/.local/lib/python3.8/site-packages/clikit/api/command/command.py:171 in _do_handle
       169│         handler_method = self._config.handler_method
       170│
     → 171│         return getattr(handler, handler_method)(args, io, self)
       172│
       173│     def __repr__(self):  # type: () -> str

  14  ~/.local/lib/python3.8/site-packages/cleo/commands/command.py:92 in wrap_handle
        90│         self._command = command
        91│
     →  92│         return self.handle()
        93│
        94│     def handle(self):  # type: () -> Optional[int]

  13  ~/.local/lib/python3.8/site-packages/poetry/console/commands/show.py:82 in handle
        80│         solver.provider.load_deferred(False)
        81│         with solver.use_environment(self.env):
     →  82│             ops = solver.solve()
        83│
        84│         required_locked_packages = set([op.package for op in ops if not op.skipped])

  12  ~/.local/lib/python3.8/site-packages/poetry/puzzle/solver.py:65 in solve
        63│         with self._provider.progress():
        64│             start = time.time()
     →  65│             packages, depths = self._solve(use_latest=use_latest)
        66│             end = time.time()
        67│

  11  ~/.local/lib/python3.8/site-packages/poetry/puzzle/solver.py:233 in _solve
       231│
       232│         try:
     → 233│             result = resolve_version(
       234│                 self._package, self._provider, locked=locked, use_latest=use_latest
       235│             )

  10  ~/.local/lib/python3.8/site-packages/poetry/mixology/__init__.py:7 in resolve_version
       5│     solver = VersionSolver(root, provider, locked=locked, use_latest=use_latest)
       6│
     → 7│     return solver.solve()
       8│

   9  ~/.local/lib/python3.8/site-packages/poetry/mixology/version_solver.py:84 in solve
        82│             while next is not None:
        83│                 self._propagate(next)
     →  84│                 next = self._choose_package_version()
        85│
        86│             return self._result()

   8  ~/.local/lib/python3.8/site-packages/poetry/mixology/version_solver.py:374 in _choose_package_version
       372│         if locked is None or not dependency.constraint.allows(locked.version):
       373│             try:
     → 374│                 packages = self._provider.search_for(dependency)
       375│             except ValueError as e:
       376│                 self._add_incompatibility(

   7  ~/.local/lib/python3.8/site-packages/poetry/puzzle/provider.py:131 in search_for
       129│
       130│         if dependency.is_vcs():
     → 131│             packages = self.search_for_vcs(dependency)
       132│         elif dependency.is_file():
       133│             packages = self.search_for_file(dependency)

   6  ~/.local/lib/python3.8/site-packages/poetry/puzzle/provider.py:163 in search_for_vcs
       161│             return [self._deferred_cache[dependency]]
       162│
     → 163│         package = self.get_package_from_vcs(
       164│             dependency.vcs,
       165│             dependency.source,

   5  ~/.local/lib/python3.8/site-packages/poetry/puzzle/provider.py:202 in get_package_from_vcs
       200│             revision = git.rev_parse(reference, tmp_dir).strip()
       201│
     → 202│             package = cls.get_package_from_directory(tmp_dir, name=name)
       203│             package._source_type = "git"
       204│             package._source_url = url

   4  ~/.local/lib/python3.8/site-packages/poetry/puzzle/provider.py:285 in get_package_from_directory
       283│         cls, directory, name=None
       284│     ):  # type: (Path, Optional[str]) -> Package
     → 285│         package = PackageInfo.from_directory(path=directory).to_package(
       286│             root_dir=directory
       287│         )

   3  ~/.local/lib/python3.8/site-packages/poetry/inspection/info.py:147 in to_package
       145│             )
       146│
     → 147│         package = Package(
       148│             name=name,
       149│             version=self.version,

   2  ~/.local/lib/python3.8/site-packages/poetry/core/packages/package.py:51 in __init__
        49│         Creates a new in memory package.
        50│         """
     →  51│         super(Package, self).__init__(
        52│             name,
        53│             source_type=source_type,

   1  ~/.local/lib/python3.8/site-packages/poetry/core/packages/specification.py:19 in __init__
        17│     ):  # type: (str, Optional[str], Optional[str], Optional[str], Optional[str], Optional[List[str]]) -> None
        18│         self._pretty_name = name
     →  19│         self._name = canonicalize_name(name)
        20│         self._source_type = source_type
        21│         self._source_url = source_url

  TypeError

  expected string or bytes-like object

  at ~/.local/lib/python3.8/site-packages/poetry/core/utils/helpers.py:24 in canonicalize_name
       20│ _canonicalize_regex = re.compile("[-_]+")
       21│
       22│
       23│ def canonicalize_name(name):  # type: (str) -> str
    →  24│     return _canonicalize_regex.sub("-", name).lower()
       25│
       26│
       27│ def module_name(name):  # type: (str) -> str
       28│     return canonicalize_name(name).replace(".", "_").replace("-", "_")

[sinoroc]

From a first quick look, it seems like renpy is not following the packaging standards. This might explain why it is not automatically installable.

[kaldown]

Hello from January of 2021. Same issue appears after updating whole bunch of dependencies. No way to find "why"

[godovdm]

Unfortunately I have the same probem, got requests package and tried to install it via poetry from git with no luck.

[sinoroc]

As I said, renpy is not following common project structures and packaging techniques. It is not easy to install, and poetry just can't install it. It does not seem to have a setup.py or pyproject.toml file at the root.

@kaldown

No way to find "why"

I explained why.

@godovdm

Are you sure you are having exactly the same issue? Are you also installing to install an unusually-packaged dependency such as renpy?

[kaldown]

@sinoroc i do not use renpy.
Common libraries like celery, flask, alchemy...

Removing poetry env and installing from scratch fixed the problem.

Wish there will be time when package managers will just work

[sinoroc]

Wish there will be time when package managers will just work

If you are certain the issue lies in poetry, it would be helpful to get some actual report with meaningful debug info. Anyway, glad you fixed your virtual environment.

[kaldown]

How could I get debug info if poetry just stuck?

No command is working.
Even poetry show.

Poetry.lock and pyproject.toml are correct ones. Because re-intalling poetry with the same scheme works.

What else could it be?

[sinoroc]

What else could it be?

There are many things that could happen and break a virtual environment, it's sad but it happens, nothing dramatic about it. Without knowing the true cause, no need to blame poetry or any other party.

[tebeka]

I'm using an empty environment (after poetry init) and then poetry add git+https://github.com/353solutions/storey which reproduces the bug.

pip on the other hand manages to install this dependency.

[sinoroc]

I'm using an empty environment (after poetry init) and then poetry add git+https://github.com/353solutions/storey which reproduces the bug.

I can recreate the issue.
Not sure on what poetry chokes exactly. The project linked seems to be packaged correctly. The verbose output is not really helpful.

[remram44]

I am running into this problem. If one of the dependencies uses something like:

setup(
    install_requires=['pkg @ git+https://github.com/pkg/pkg.git@12345#egg=pkg'],
    ...
)

Then poetry fails with TypeError.

If the dependency's setup.py is changed so it reads:

setup(
    install_requires=['git+https://github.com/pkg/pkg.git@12345#egg=pkg'],
    ...
)

Then it works.

Poetry simply needs to be updated for the new format from PEP-508 https://www.python.org/dev/peps/pep-0508/#examples

[sinoroc]

setup(
    install_requires=['pkg @ git+https://github.com/pkg/pkg.git@12345#egg=pkg'],
    ...
)

Then poetry fails with TypeError.

Have you tried without the #egg=pkg part? Because it looks to me like this part does not belong in a PEP508 conform notation. This part is usually there to specify the actual name of the project, but in PEP 508 the name of the project is already at the beginning before the @.

[finswimmer]

@sinoroc is absolutely right here.

• renpy doesn't have a valid package project structure. There is no setup.py or pyproject.toml within the root folder
• The example given by @remram44 isn't a valid PEP 508 string because of the #egg=pkg part.

The issue with git+https://github.com/353solutions/storey with a bit tricky. The issue here is, that the default branch for this repo is not master. It is development. At the moment poetry expects master as the default branch if nothing is given. So it checks out the master branch for dependency resolution. But this branch is nearly empty.

Handling of the default branch is discussed here: #3366. There was already a PR to fix it (python-poetry/poetry-core#143), but it has to get reverted for now (python-poetry/poetry-core#148).

fin swimmer

[remram44]

Have you tried without the #egg=pkg part?

Yes I had tried that first.

[finswimmer]

Hello @remram44,

could you please open a new issue with a reproducible example if you think this is a problem?

fin swimmer

[remram44]

A new issue with the same title and the same content? Would you leave that one open?

[finswimmer]

Hey @remram44,

it doesn't sound like your issue have the same root cause. So it's perfectly fine to open a separate issue. Especially a reproducible would save us a lot of time.

fin swimmer

[brocef]

I just ran into this issue on poetry v1.1.12 and the solution for me was to make sure I add the repo with a tag/revision specified.

Instead of poetry add git+ssh://[email protected]/orgname/repo.git
I did poetry add git+ssh://[email protected]/orgname/repo.git#main (main being the main branch, not master)

and it succeeded

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.