[INT-442] Poetry

[bbonenfant]

Switch from setup.py to poetry/pyproject.toml

This changes the following things:

• No more need for a setup.py file, so it gets deleted. Building the wheel is done through poetry build and the the dependencies are tracked with pyproject.toml and poetry.lock
• There's also no need for the MANIFEST.in file as poetry does all of that in the pyproject.toml file. There should probably be a review of this to ensure that the proper things get included in the package that were previously specified in the manifest.
• poetry installs the virtual environment at .venv instead of venv. No way I know around this unfortunately.
• I switched how the __version__ value gets specified. The modern way to do this is to look at the metadata of the installed package -- this functionality is built into the language in python >=3.8, but there is backport (importlib-metadata) for older versions.
  • On this point, poetry must have the version specified in the pyproject.toml so there should probably be a discussion on how this meshes with theversion.json file.
• Had to make some tweaks to get CI working (but surprisingly fewer tweaks than I was expecting)
• I changed the author of the package on whim to Pachyderm <[email protected]>, but I can revert to JD if that was a mistake.

Follow ons:

• There should be a review and sanitization of the dependency pinning. In general testing dependencies should be pinned and the ^ dependency selector should be used over >/=> for package dependencies.

[CLAassistant]

All committers have signed the CLA.

[albscui]

LGTM!

Although I don't know poetry that well, this looks really useful!

[albscui]

Does this install the dev dependencies as well?

[bbonenfant]

Yep! When run like this on the repo poetry will install all dependencies, including dev dependencies.

[smalyala]

Just to confirm, if we want to bump the python-pachyderm version, we need to update it in pyproject.toml and version.json. As of right now, it seems like the python-pachyderm version in version.json is only being used in setup.py (and in test_version.py). Correct me if I'm wrong, but we can effectively remove the python-pachyderm line from version.json and instead use the version number from pyproject.toml for version tests right?

[smalyala]

Let's use the email [email protected] instead. @albscui thoughts?

[bbonenfant]

I grabbed this from this issue on a whim. Happy to change it to the correct one

[smalyala]

Great! This integrations email is the one we're using for managing all integrations-related projects moving forward.

[smalyala]

Does this read directly from pyproject.toml?

[bbonenfant]

Nope, packages keep their official version in an associated METADATA file that these modules attempt to read (this is how pip can report the version of installed packages without a lock file).

[smalyala]

What does this file do?

[bbonenfant]

poetry by default tries to create all you virtual environments in your home directory, this just tells poetry to create the virtual environment in the project

[smalyala]

Neat!

[smalyala]

LGTM, just one question up above