Packages at relative paths not found #1518
Comments
|
Before digging into getting this style of PEX build working, can you try the much simpler PEP-517 strategy (I.E.: just let Poetry build it) below and report back?: This should work for any Poetry or other PEP-517 built project. Pex in fact dogfoods this for itself (it uses Flit instead of Poetry): |
|
|
Urg. This should mean Pip also fails? Can you try |
|
Yup, unfortunately: |
|
Pre-emptively - seems relevant: https://stackoverflow.com/questions/64807176/pip-install-with-relative-paths-in-poetry |
|
Ahh ok, so Poetry supports relative paths but not pip |
|
Well, it does apparently but behind a flag. Can you try?: |
|
If that works, I can probably expose an equivalent flag for that in Pex. |
|
Oh interesting, good to know. Same error: |
|
Hrm. OK. I don't think there is much I'm going to be able to do here. Is this workflow acceptable / does it even work for you?: I'm relying here on Poetry hopefully building all local wheels it needs for the current project and plopping them in dist. I tested against poetry itself, but that just has the 1 project. I don't have my hands on a multi-project structure to test this quickly for myself. |
|
Ok, cloned this: https://github.com/ya-mori/python-monorepo And: |
|
Afaict you'll either need to build all needed local wheels yourself and then combine those with the exported poetry lock - lots of fiddly there. Otherwise it seems like #1361 would work if implemented since it appears to be the case that |
|
This gives the same error: rm -rf dist/
poetry build -f wheel
pex dist/*.whl --script=lexio-db --output-file=lexio-db.pex
# pip._vendor.pkg_resources.RequirementParseError: Invalid URL: ../lib/database_seedingI'll see if building all the local wheels could work. If it doesn't I'll probably end up refactoring the system around it to use a regular installation method. Thanks so much for digging in @jsirois! |
|
Aha, here is the issue: Poetry is building invalid wheels via: |
|
I'll see if this ug has been filed. I do hope this is only the case for the |
|
Ok, @jdrake the only way to get this working sanely in Pex will be if #1361 gets implemented; otherwise, Poetry is breaking too many PEPs to work with without writing a bunch of special case code to deal with Poetry projects. I'm going to close as won't fix in favor of #1361. Please speak up though if you have ideas on other ways for Pex to handle this. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I have a monorepo and am using Poetry to manage all the individual packages. (I'm not using Pants for this for other reasons.)
Packages depend on a mix of regular external libraries in pypi and local packages. For example:
From here, the
ns_database_seedingpackage depends on other internal packages and use path links too.Next I export the requirements:
poetry export -f requirements.txt --without-hashes -o requirements.txtwhich gives something like:
To build the pex I run:
But that fails with:
It seems like pex (or pip under the hood) is picking up on the pyproject.toml files for internal dependencies (via pip hook) but misinterpreting the relative paths. They are in each pyproject.toml file and poetry.lock file relative to that project, but it seems like pex or pip is interpreting them as relative to some virtualenv or current directory or something.
I am using the newest versions of pip, poetry, and pex.
The text was updated successfully, but these errors were encountered: