Packaging: Use of pyproject.toml and move to minimal setup.py for modernization

[fmalatino]

Description
This PR moves the metadata and configuration for the package to a pyproject.toml file, as is prescribed by Deprecate legacy setup.py develop mechanism for pip install --editable #11457. The previous setup.py file is now minimal. `

How Has This Been Tested?
Tested using the current CI workflow

Checklist:

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• Any dependent changes have been merged and published in downstream modules
• New check tests, if applicable, are included

[romanc]

Two questions:

1. Is there a way to move the non-optional dependencies also to the pyproject.toml and get rid of setup.py?
2. Did you test local developer workflows, e.g. installing NDSL (and gt4py/dace) editable?

[fmalatino]

Two questions:

1. Is there a way to move the non-optional dependencies also to the pyproject.toml and get rid of setup.py?
2. Did you test local developer workflows, e.g. installing NDSL (and gt4py/dace) editable?

@romanc

1. Unless we want to pull dace and gt4py from their respective repositories, we will need to continue to use the dynamic dependencies definitions
2. Yes I tested this by comparison to what is done currently when running pip install -e .[extras] on develop and confirming that both generate editable installations. In both cases, to achieve and editable install of either gt4py and dace one would have to go to their respective directories and run a pip install -e . with either install method.

[romanc]

Thanks Frank, that sounds good.

Another question: what do you think about moving the configuration from setup.cfg to pyproject.toml too?

[oelbert]

👍

[fmalatino]

Thanks Frank, that sounds good.

Another question: what do you think about moving the configuration from setup.cfg to pyproject.toml too?

@romanc, given the current issues between Flake8 and parsing its configuration from a pyproject.toml file, I vote to leave it as is, also given that we are holding on to the setup.py file as well. When we are able to get rid of the setup.py file, I think it will make sense to also get rid of the setup.cfg file. Thoughts?

[romanc]

Thanks Frank, that sounds good.
Another question: what do you think about moving the configuration from setup.cfg to pyproject.toml too?

@romanc, given the current issues between Flake8 and parsing its configuration from a pyproject.toml file, I vote to leave it as is, also given that we are holding on to the setup.py file as well. When we are able to get rid of the setup.py file, I think it will make sense to also get rid of the setup.cfg file. Thoughts?

I'm unaware about problems between flake8 and reading the configfile. That sounds weird to me given that pyproject.toml is the new standard. But yeah, if there are issues, we can iterate and do follow-up PRs.

Is this ready to be merged/reviewed? Asking because the PR is still marked as draft...

[romanc]

Two more comments:

1. Re flake8: maybe double check. The version was updated (3.x -> 7.x) two weeks ago in PR Update pre-commit hooks #172. Maybe that helped?
2. I think it would make sense to bump setuptools to >=77.0.3 and then update the license / license-files information as suggested in the official guide. This would allow to cleanly include the license information of ndsl/viz/fv3.

[fmalatino]

Two more comments:

1. Re flake8: maybe double check. The version was updated (3.x -> 7.x) two weeks ago in PR Update pre-commit hooks #172. Maybe that helped?
2. I think it would make sense to bump setuptools to >=77.0.3 and then update the license / license-files information as suggested in the official guide. This would allow to cleanly include the license information of ndsl/viz/fv3.

I will continue to dig on this, my focus is currently on finishing out the analogous changes in pace. Once I am ready for review I will change the status of this PR, I apologize for the confusion.

[romanc]

I've been reading up on python packing and I agree, this is the most standard way add local package dependencies. It will support all current workflows and prep us for the future to come.

This is starting to look nice. To be clear, can we get rid of setup.cfg now? Otherwise just some minor suggestions inline.

(And sorry for the un-asked review. I was just monitoring activity and thought I'd share my thoughts / endorsement for this move to pyproject.toml.

[romanc]

Coming along nicely! 🚀

[romanc]

Nice! Thanks for bringing us up to speed with recent python config developments 🚀