Un-pin hatchling.

[jsirois]

Although hatchling fixed a few issues that broke Pex packaging, it has
since more strictly adhered to PEP-621 which breaks the existing project
metadata customization scheme. Update the scheme to conform to PEP-621
strictures. Also update the tox configuration to work with modern
hatchling.

Picks up from #2403 by @ofek

[jsirois]

I think this effective move of the project metadata for requires-version and urls from below to here helps underline how the PEPs missed the boat here by not just saying dynamic trumps static, allowing the dynamic code to run; yet use standard static metadata as part of its inputs. This minor hoop I jumped through seems gratuitous on behalf of the spec. @ofek I know you're just doing your job implementing the spec.

[ofek]

Although it is true that I'm trying to adhere to the spec, I was actually the first to implement it and was part of the design! I definitely understand your perspective and I can't go through the threads right now but there was a good reason for doing so. I think it had something to do with resolvers not being able to depend on data and thus being forever unable to use the metadata directly within source distributions and therefore require building during resolution in many cases.

[ofek]

This is great, thanks for fixing the tox issue! I was thinking of somehow defining the environment variable conditionally outside of tox but your solution is much better. I asked the maintainer Bernát in Discord and it looks like there is a build step that happens first which all environments then use. Just a heads up, this would have worked out of the box with Hatch because builds happen with access to the environment variables of the individual environments 😄

[jsirois]

this would have worked out of the box with Hatch

Yeah, I find tools like pdm, poetry, hatch, etc do too much, which is exactly what most people like. I just want an sdist / wheel builder and I like the extreme simplicity of tox / nox / etc. Hatchling was attractive because of the clean means to plug in; that's why I switched to it after having some issues with setuptools leaking .egg-info cruft.

[ofek]

I agree that Poetry does too much/hard to determine the behavior. Since I am currently finishing up the slides for my talk at PyCon, have you seen this comparison? I added the page a few months ago: https://hatch.pypa.io/latest/why/#environment-management

Here is an example of the difference in config:

tox.ini

hatch.toml

[tox] envlist = py{27,38}-{42,3.14} py{38,39}-{9000}-{foo,bar} [testenv] usedevelop = true deps = coverage[toml] pytest foo: cryptography commands = pytest {posargs:tests} setenv = 3.14: PRODUCT_VERSION=3.14 42: PRODUCT_VERSION=42 9000: PRODUCT_VERSION=9000 {foo,bar}: EXPERIMENTAL=true

[[envs.default.matrix]] python = ["2.7", "3.8"] version = ["42", "3.14"] [[envs.default.matrix]] python = ["3.8", "3.9"] version = ["9000"] feature = ["foo", "bar"] [envs.default] dependencies = [ "coverage[toml]", "pytest", ] [envs.default.scripts] test = "pytest {args:tests}" [envs.default.overrides] matrix.version.env-vars = "PRODUCT_VERSION" matrix.feature.env-vars = "EXPERIMENTAL=true" matrix.feature.dependencies = [ { value = "cryptography", if = ["foo"] }, ]

[jsirois]

@ofek I haven't. That said, I really detest the current state of the PyPA. Good people, horrible system of doing business. There is a new tool and a new packaging PEP a few times a year. Until PyPA merges back into core Python and Python owns shipping a toolset and not just the cpython interprter (in other words one official tool set), I will not thrash. Hopefully someone with pull cries uncle and this merger of PyPA back into core python happens and the PSF strong-arms the platinum sponsors to fork over ~10 million or whatever it takes to dedicate the resources to kick out this sort of thing in a ~short time - say 5 years.

[ofek]

Jotting down my thoughts while I have a few minutes, I really appreciate your openness for discussion!

• I am actively trying to provide what users would consider the main toolset with my development of Hatch/Hatchling.
• I haven't witnessed the entire history firsthand but what you're saying is the opposite of what CPython folks want, indeed everything that is perceived to have improved with packaging over the past decade or more was in spite of CPython. Most people there quite simply I'm sorry to say don't care about packaging. There are a handful that care very much like Brett Cannon but they are few in number. It's a simplification but what happened a while back was essentially an effort to improve packaging posed to CPython people and it wasn't taken very seriously so then the Anaconda ecosystem was created to solve the situation. Everything since then has happened by folks outside of CPython (although a few maintainers overlap) which everyone now refers to as PyPA.
• I understand that it might seem like there are many backward incompatible PEPs every year but from my experience that's not the case and churning is pretty much over, although I'm interested in hearing your feedback on my perception. I know you're aware but I feel the need to say that standardization/RFCs/PEPs is required for interoperability. For example, you can't really standardize metadata parsing from the execution of a setup.py file as Dependabot is written in Ruby, Visual Studio Code in TypeScript, etc. Other ecosystems like Rust also have a standardization process and sometimes for the betterment of the community choose controversial paths, e.g.: RFC: Make Cargo respect minimum supported Rust version (MSRV) when selecting dependencies rust-lang/rfcs#3537

edit: not really you, but generally speaking when people talk about the sorry state of Python packaging currently it reminds me of the paper Prevalence-induced concept change in human judgment

[jsirois]

I haven't witnessed the entire history firsthand but what you're saying is the opposite of what CPython folks want, indeed everything that is perceived to have improved with packaging over the past decade or more was in spite of CPython.

I get the same impression. What I'm pointing out is the effort to work around the problem presented by CPython suffers from not confronting the problem head on and fixing that core problem. Forking off the PyPA has allowed (slow) forward progress, but at the cost of the necessary evil (in my opinion) of having to focus on standards generation instead of just implementing one. In other words, without the official backing of CPython you can't ship 1 toolset alongside CPython releases. You must, instead, try to cajole the organic ecosystem of tools onto a set of standards. You appear to accept the rules of that game. I question those rules as being the right ones.

My personal number one rule though is you can't have an opinion unless you have skin in the game. Here I'm in an awkward spot. I have skin in the game being a middleman between the current state of PyPA and Python ecosystem users experiencing packaging problems, but I don't have skin in the game in terms of getting involved in the PyPA / CPython / PSF and trying to change what I see as a misguided direction. That would require many years of effort that I'm not willing to put in at this point in my life. At 52 with my priority being rock climbing, I only have time to help finish Wickman's original Pex vision started in 2010. So I'm definitely solidly in the bitching category by my own standards, and you shouldn't listen to people who are doing that!

I understand that it might seem like there are many backward incompatible PEPs every year but from my experience that's not the case and churning is pretty much over,

Oh, I have no issue with the PEPs individually, and I don't see them as being backward incompatible. As mentioned above I disagree with the PEPs on a more basic level. They encourage many tools to be compatible and I don't want many tools. I just want one official toolset with the PEPs maybe serving as public design doc records, but not serving as the spring board for 1000 tools.

[ofek]

Thank you!