Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Is there a 3.13 nogil container? #947

Open
impredicative opened this issue Aug 10, 2024 · 9 comments
Open

Is there a 3.13 nogil container? #947

impredicative opened this issue Aug 10, 2024 · 9 comments

Comments

@impredicative
Copy link

impredicative commented Aug 10, 2024

I want a 3.13 nogil container, also with all optimizations, etc. I assume that the current 3.13 containers use the GIL.

@tianon
Copy link
Member

tianon commented Aug 12, 2024

We don't have a "nogil" image currently. For 3.13 we're using the default compilation settings, which AFAIK still results in the GIL. Unfortunately, compiling Python is pretty heavy, so expanding our current support matrix to include new "nogil" variants is also going to be a tough sell. 🙈

@tianon
Copy link
Member

tianon commented Aug 12, 2024

I'm not sure how you got there from what I said -- perhaps I can clarify: we do compile Python (currently ~42 times per architecture, in fact, and we support ~10 architectures where compilation is necessary), and it's very heavy to do so, and adding more variants would require us to do so even more times, so is not something we're willing to consider unless/until it is an officially recommended configuration (ideally the default configuration, but we'd be willing to consider an official statement along the lines of "everyone should try this out").

@impredicative
Copy link
Author

nogil is the future of Python. It doesn't make sense to not support it. If this request seems strange, that may be so only because I am the first person to ask for it. Rest assured, there will be many more who will progressively ask for it. The pressure will build. This request is easy to reject now, but only now.

@tianon
Copy link
Member

tianon commented Aug 12, 2024

I agree that nogil is the future of Python, and there's nothing strange about the request. If I were completely opposed to the idea in general, I probably would've closed the issue already. It's more a matter of resourcing/prioritization, not desire/understanding. If the demand were actively higher, it would also be a lot easier to justify the added resources/maintenance spend -- the best way for folks to signal that is to react with 👍 on the top post in this thread.

To put that another way, my answer is more accurately "not yet", not "never".

If you'd like to help with the work, figuring out what changes are necessary to our current Dockerfiles would be a really useful first step.

@ldeluigi
Copy link

An experimental JIT-enabled build would also be great to have

@sekrause
Copy link

Instead of prodiving a whole second set of "nogil" Docker images the free-threaded executable could also be part of the normal images just like it is done with the official Windows and macOS installers:

CPython now has experimental support for running in a free-threaded mode, with the global interpreter lock (GIL) disabled. This is an experimental feature and therefore is not enabled by default. The free-threaded mode requires a different executable, usually called python3.13t or python3.13t.exe. Pre-built binaries marked as free-threaded can be installed as part of the official Windows and macOS installers, or CPython can be built from source with the --disable-gil option.

https://docs.python.org/3.13/whatsnew/3.13.html#free-threaded-cpython

So we would have /usr/local/bin/python3.13t in every image and call this to use the free-threaded mode.

Unfortunately that still means that all of Python has to be compiled twice.

@impredicative
Copy link
Author

impredicative commented Aug 27, 2024

@sekrause But that would break a lot of tooling that expects Python to be at the prior file path.

@sekrause
Copy link

But that would break a lot of tooling that expects Python to be at the prior file path.

The normal paths would still be there, but point to the normal GIL version. So if you don't adjust your tooling everything will be as before.

@abebus
Copy link

abebus commented Sep 2, 2024

If anyone seeking for some basic image to play with, I've made one: https://github.com/abebus/free-threaded-python-docker-image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants