Poetry add fails if there is a mercurial repo in src

[tiktuk]

• I am on the latest Poetry version.
• I have searched the issues of this repo and believe that this is not a duplicate.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: Debian GNU/Linux 9.6 (stretch)
• Poetry version: 1.0.5

Issue

I needed to install a package from a Mercurial repo and since Poetry doesn't support Mercurial I installed it using Pip. I did it in editable mode using pip -e. After that poetry add <other package> fails with

$ poetry add <package>

[CalledProcessError]
Command '['git', '--git-dir', '/home/user/.virtualenvs/venv/src/myhgpackage/.git', '--work-tree', '/home/user/.virtualenvs/venv/src/myhgpackage', 'rev-parse', 'HEAD^{commit}']' returned non-zero exit status 128.

So it looks like Poetry assumes all repos in src are git repos even though that is not necessarily the case.

[colatkinson]

I actually have a somewhat related issue, although I can move this to a separate issue if it's deemed off-topic.

I ran into the same symptom (i.e. git rev-parse failing), but a different root cause. In my case, it was due to a git repo using a nonstandard setup.py, which actually had multiple calls to setuptools.setup(). I have a minimal reproduction here. While I'm not sure if this is something the maintainers want to even attempt to support, I thought it might be useful to share this information regardless.

I also did a bit of diagnostic work, though I'll be the first to admit I'm not familiar at all with the codebase. When running poetry install, the Installer class takes stock of what packages are already installed via the InstalledRepository class.

InstalledRepository is responsible for iterating over all of the currently installed packages. It first checks if the package was installed in the standard location, and then checks if it was installed within src. If it is, then as @tiktuk mentioned, it is assumed that src/[pkg_name] is a git repo.

For this case, would it make sense to check if the .git directory in question exists, and if not fall back to treating the package type as directory? Though I'm not sure what differences in handling there are between VCS and directory packages, so there could be other effects.

---

The failure in my case actually ends up a bit more convoluted. The code uses importlib.metadata to retrieve the package name. Since the package is not installed at src/[pkg_name], the git command will fail with the same message.

Changing how the path is calculated could help, but even then multiple packages would end up pointing to the same repo, which seems strange. I think a better error message could be helpful, even if the behavior is kept as-is.

[finswimmer]

Hello,

I recognized this when working on #2242.

It can be fixed in this part:

poetry/poetry/repositories/installed_repository.py

Lines 58 to 72 in bf331a5

	try:
	path.relative_to(src_path)
	from poetry.vcs.git import Git
	git = Git()
	revision = git.rev_parse("HEAD", src_path / package.name).strip()
	url = git.remote_url(src_path / package.name)
	package.source_type = "git"
	package.source_url = url
	package.source_reference = revision
	except ValueError:
	package.source_type = "directory"
	package.source_url = str(path.parent)

by catching CalledProcessError. I already implement this in my Feature-PR. I'm not sure if we release another bug fix before the next minor release. If so I could add this fix already there. What do you think @sdispater ?

fin swimmer

[KelSolaar]

I just came across a related one, I added a git repo to my pyproject.toml file:

colour = { git = "https://github.com/colour-science/colour.git", branch = "develop" }

poetry install worked fine, but subsequent calls to poetry update are failing:

(gamut-mapping-ramblings-D9duBHn--py3.7) Kali:gamut-mapping-ramblings kelsolaar$ poetry update

[CalledProcessError]
Command '['git', '--git-dir', '/Users/kelsolaar/Library/Caches/pypoetry/virtualenvs/gamut-mapping-ramblings-D9duBHn--py3.7/src/colour-science/.git', '--work-tree', '/Users/kelsolaar/Library/Caches/pypoetry/virtualenvs/gamut-mapping-ramblings-D9duBHn--py3.7/src/colour-science', 'rev-parse', 'HEAD']' returned non-zero exit status 128.

The issue comes from the fact that the python package and the repo are different thus:

(gamut-mapping-ramblings-D9duBHn--py3.7) Kali:gamut-mapping-ramblings kelsolaar$ ll /Users/kelsolaar/Library/Caches/pypoetry/virtualenvs/gamut-mapping-ramblings-D9duBHn--py3.7/src/colour-science/.git
ls: /Users/kelsolaar/Library/Caches/pypoetry/virtualenvs/gamut-mapping-ramblings-D9duBHn--py3.7/src/colour-science/.git: No such file or directory
(gamut-mapping-ramblings-D9duBHn--py3.7) Kali:gamut-mapping-ramblings kelsolaar$ ll /Users/kelsolaar/Library/Caches/pypoetry/virtualenvs/gamut-mapping-ramblings-D9duBHn--py3.7/src/colour-science/
ls: /Users/kelsolaar/Library/Caches/pypoetry/virtualenvs/gamut-mapping-ramblings-D9duBHn--py3.7/src/colour-science/: No such file or directory

But:

(gamut-mapping-ramblings-D9duBHn--py3.7) Kali:gamut-mapping-ramblings kelsolaar$ ll /Users/kelsolaar/Library/Caches/pypoetry/virtualenvs/gamut-mapping-ramblings-D9duBHn--py3.7/src/colour/
total 568
-rw-r--r--   1 kelsolaar  staff   132K May 10 11:23 BIBLIOGRAPHY.bib
-rw-r--r--   1 kelsolaar  staff   522B May 10 11:23 CHANGES.rst
-rw-r--r--   1 kelsolaar  staff   3.4K May 10 11:23 CODE_OF_CONDUCT.md
-rw-r--r--   1 kelsolaar  staff   577B May 10 11:23 CONTRIBUTING.rst
-rw-r--r--   1 kelsolaar  staff   4.7K May 10 11:23 CONTRIBUTORS.rst
-rw-r--r--   1 kelsolaar  staff   1.5K May 10 11:23 GRANTS
-rw-r--r--   1 kelsolaar  staff   1.5K May 10 11:23 LICENSE
-rw-r--r--   1 kelsolaar  staff    41K May 10 11:23 README.rst
-rw-r--r--   1 kelsolaar  staff    31K May 10 11:23 SPONSORS.rst
-rw-r--r--   1 kelsolaar  staff   649B May 10 11:23 THIRD_PARTY
-rw-r--r--   1 kelsolaar  staff   5.4K May 10 11:23 TODO.rst
drwxr-xr-x  28 kelsolaar  staff   896B May 10 11:23 colour
drwxr-xr-x  40 kelsolaar  staff   1.3K May 10 11:23 docs
-rw-r--r--   1 kelsolaar  staff   3.8K May 10 11:23 pyproject.toml
-rw-r--r--   1 kelsolaar  staff   2.0K May 10 11:23 requirements.txt
-rw-r--r--   1 kelsolaar  staff   4.9K May 10 11:23 setup.py
-rw-r--r--   1 kelsolaar  staff    14K May 10 11:23 tasks.py
drwxr-xr-x   6 kelsolaar  staff   192B May 10 11:23 utilities
drwxr-xr-x   3 kelsolaar  staff    96B May 10 11:24 pip-wheel-metadata
drwxr-xr-x   7 kelsolaar  staff   224B May 10 11:24 colour_science.egg-info

It relates to #1417

[donkopotamus]

I'm observing this also in poetry-1.0.4 and poetry-1.0.5 when attempting to install a project that contains a git package. The pyproject.toml has:

some_package = { git = "ssh://[email protected]/some-user/some_package.git", branch="release/some-release" }

and poetry install (from an msys2 shell) cause an error when it is (presumably) trying to determine the sha from the branch name:

...
Command '['git', '--git-dir', 'c:/users/some-user/appdata/local/temp/pypoetry-git-randomchars/.git', '--work-tree', 'c:/users/some-user/appdata/local/temp/pypoetry-git-randomchars', 'rev-parse', u'release/some-release^{commit}']' returned non-zero exit status 128

Poetry 1.0.3 is working fine.

Addendum

My problem was in fact due to #2667 (which #2668 proposes to fix)