Add an upper bound to Requires-Python?

[cjolowicz]

By default, Poetry adds an upper bound to Requires-Python. We removed this upper bound in cjolowicz/cookiecutter-hypermodern-python-instance#743. The reasons for this move are explained here:

• https://iscinumpy.dev/post/bound-version-constraints/ (see the section titled Pinning the Python version is special)
• https://discuss.python.org/t/requires-python-upper-limits/12663

Poetry does not allow adding a dependency with a Python version cap to a project without one. There is an exception to this rule, though: Such dependencies can be added with an environment marker that constrains the Python version. While this provides a workaround, there are two problems with it:

1. It's cheating. The workaround means that your package will be installed on, say, Python 4 without the offending dependencies. But if your package truly depends on them, then it won't work on Python 4. So capping Python <4 is, sadly, the correct thing to do.
2. It will most definitely trip up users of this project template, every time they add another package with a version cap to their dependency tree.

So it appears like there's no way around Python version caps for us? We depend on multiple packages that have them (isort, darglint, mdit-py-plugins via myst-parser, urllib3 via sphinx and safety).

The alternative would be to accept the cons for the sake of limiting further spread of a harmful practice (and maybe get isort, darglint, mdit-py-plugins, and urllib3 to drop their caps). But it does feel like fighting windmills, somewhat. @henryiii, any ideas?

See #1148

[henryiii]

I'd first try to contact the packages that are adding the upper limit, and see if you can get them to remove them. One of the reasons for writing the article was to have a link to help convince the packages doing this to drop it.

Second, why does your runtime environment care about things like isort? You should be using it as an app, say via pre-commit, nox/tox, pipx, or something else, and never installing it as a library into a shared environment - applications are not designed for that.

Third, you might consider PDM, which is a more modern tool that follows PEP standards much better than Poetry does. Though it also combines your Python-Requires metadata setting with the lock limits like Poetry does, which IMO is completely wrong.

[henryiii]

Ahh, you are dumping them into dev requirements. I'd go with pre-commit, and leave them out of the developers environment. You are increasing your surface area for conflicts exponentially by trying to install all of them into a single environment. They don't import each other, so they shouldn't be installed with each other. You can use pipx run to run them directly, or always/only run them through pre-commit. Applications are not designed to run in shared environments. As you see in that article, black and PyTorch were incompatible! So your "dev" environment should only contain a minimum of things you need for development, not stylistic stuff that just duplicates pre-commit.

[henryiii]

See https://scikit-hep.org/developer and https://github.com/scikit-hep/cookie for suggestions and examples.

[cjolowicz]

Thanks @henryiii for providing your thoughts!

As an interim measure, I'll be restoring the upper bound on the Python version.

Let's continue this discussion and evaluate better approaches for the future.