Remove support for threads-less builds

[pitrou]

BPO	31370
Nosy	@brettcannon, @gpshead, @pitrou, @tiran, @bitdancer, @briancurtin, @pmp-p, @serhiy-storchaka, @kb-1000
PRs	bpo-31370: Remove support for threads-less builds #3385 bpo-31370: Simplify run_tests.py #3482 Restore dummy_threading and _dummy_thread, but deprecate them (bpo-31370) #3648 Trivial cleanups following bpo-31370 #3649 bpo-31370: Remove references to threadless builds #8805

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2017-11-03.06:42:40.326>
created_at = <Date 2017-09-06.21:10:49.815>
labels = ['interpreter-core', 'type-feature', '3.7', 'build']
title = 'Remove support for threads-less builds'
updated_at = <Date 2022-03-09.09:35:58.994>
user = 'https://github.com/pitrou'

bugs.python.org fields:

activity = <Date 2022-03-09.09:35:58.994>
actor = 'vstinner'
assignee = 'none'
closed = True
closed_date = <Date 2017-11-03.06:42:40.326>
closer = 'berker.peksag'
components = ['Build', 'Interpreter Core']
creation = <Date 2017-09-06.21:10:49.815>
creator = 'pitrou'
dependencies = []
files = []
hgrepos = []
issue_num = 31370
keywords = ['patch']
message_count = 29.0
messages = ['301518', '301520', '301521', '301539', '301543', '301559', '301595', '301596', '301805', '301964', '301968', '301971', '301973', '301980', '301987', '301988', '302474', '302476', '302484', '302492', '326681', '348490', '408243', '408257', '408501', '408508', '408518', '408575', '408594']
nosy_count = 11.0
nosy_names = ['brett.cannon', 'gregory.p.smith', 'pitrou', 'christian.heimes', 'Arfrever', 'r.david.murray', 'brian.curtin', 'pmpp', 'serhiy.storchaka', 'kb1000', 'Steve Gomez']
pr_nums = ['3385', '3482', '3648', '3649', '8805']
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue31370'
versions = ['Python 3.7']

[pitrou]

The --without-threads option has a lot of repercussions on the code base, forcing us to maintain specific paths for builds with threads disabled, for no useful purpose.

[pitrou]

See discussion in https://mail.python.org/pipermail/python-dev/2017-September/149156.html

[pitrou]

Question: should we keep Modules/Setup.config.in? It's going to become empty:
https://github.com/python/cpython/blob/master/Modules/Setup.config.in

[vstinner]

The PEP-11 should be updated just to mention that platforms without threading support are not supported in Python 3.7.

[vstinner]

We had a buildbot testing --without-threads, but it's gone. I just removed its unused class from the buildbot configuration:

python/buildmaster-config@091f52a

[gpshead]

I guess Setup.config.in can go. It's only purpose was for autoconf driven Setup changes.

[vstinner]

New changeset a6a4dc8 by Victor Stinner (Antoine Pitrou) in branch 'master':
bpo-31370: Remove support for threads-less builds (bpo-3385)
a6a4dc8

[vstinner]

PEP-11 was updated, the change was merged, this issue can now be closed. Thanks Antoine!

[Arfrever]

Also 'import threading' code in Tools/scripts/run_tests.py should be cleaned.

[Arfrever]

Lib/_dummy_thread.py and Lib/dummy_threading.py were deleted, but Doc/library/_dummy_thread.rst and Doc/library/dummy_threading.rst still exist.

Probably Doc/whatsnew/3.7.rst should mention that these modules were deleted.

Or maybe they should be restored and firstly deprecated, before deletion in a later version of Python.

[vstinner]

Arfrever: "Lib/_dummy_thread.py and Lib/dummy_threading.py were deleted, but Doc/library/_dummy_thread.rst and Doc/library/dummy_threading.rst still exist."

Ah, it should be deleted as well.

Arfrever: "Probably Doc/whatsnew/3.7.rst should mention that these modules were deleted."

Right.

Arfrever: "Or maybe they should be restored and firstly deprecated, before deletion in a later version of Python."

I don't think that it's worth it, but I suggest to add a warning to Python 3.6 documentation. Something like the following warning in Python 2.7 documentation of the commands module:
"Deprecated since version 2.6: The commands module has been removed in Python 3. Use the subprocess module instead."
https://docs.python.org/2/library/commands.html

[vstinner]

Oh, Arfrever wrote me on IRC that the removal of dummy_threading broke portage on Gentoo: https://bugs.gentoo.org/630730

eventlet also uses dummy_threading:

eventlet/support/dns/entropy.py:23: import dummy_threading as _threading
eventlet/support/dns/resolver.py:29: import dummy_threading as _threading

Hum, maybe we need one release with the deprecated module, and only remove it in Python 3.8?

[tiran]

+1 to restore dummy_threading and go through a proper deprecation phase in 3.7+.

-0 to restore _dummy_thread. It's an internal, private module.

[serhiy-storchaka]

Following lines needs updating:

Lib/importlib/_bootstrap.py:1135
Lib/_pydecimal.py:436
Lib/test/ssl_servers.py:11
Lib/test/support/init.py:2053
Lib/test/test_httplib.py:1080
Lib/test/test_idle.py:4
Lib/test/test_multiprocessing_main_handling.py:3
Lib/test/test_thread.py:5
Lib/test/test_threadsignals.py:8
Lib/test/test_tools/test_sundry.py:43
Lib/test/test_winreg.py:7
Lib/test/test_wsgiref.py:256
Lib/test/test_xmlrpc.py:1179

[bitdancer]

dummy_threading should definitely not have been removed, and like all the other APIs should not be removed until 2.7 is dead. Deprecating it is of course fine :)

[bitdancer]

And actually, I wouldn't be surprised if eventlet depended on the *functionality* in _dummy_threading, so you probably need to restore that, too.

[pitrou]

eventlet also uses dummy_threading

Not sure what you mean by that. eventlet uses the regular try/except import dance around "import threading":

https://github.com/eventlet/eventlet/blob/master/eventlet/support/dns/entropy.py#L20-L23
https://github.com/eventlet/eventlet/blob/master/eventlet/support/dns/resolver.py#L23-L26

As for portage, Arfrever fixed it :-)

https://gitweb.gentoo.org/proj/portage.git/commit/?id=504f66b0e25281e4465ebeceb799c3e54ff2b884

That said, I am not against bringing dummy_threading back, though it would sound rather weird to use it unconditionally...

[pitrou]

#3648 restores dummy_threading and _dummy_thread, but deprecates them.

[pitrou]

New changeset b43c4ca by Antoine Pitrou in branch 'master':
Restore dummy_threading and _dummy_thread, but deprecate them (bpo-31370) (bpo-3648)
b43c4ca

[pitrou]

New changeset 88c60c9 by Antoine Pitrou in branch 'master':
Trivial cleanups following bpo-31370 (bpo-3649)
88c60c9

[briancurtin]

New changeset eef0596 by Brian Curtin (Zackery Spytz) in branch 'master':
bpo-31370: Remove references to threadless builds (bpo-8805)
eef0596

[kb-1000]

Unfortunately, this also leads to less good LTO if you use a single static executable, similar to what Tools/freeze produces but including libpython itself. I'm currently trying to make LLVM do that, but since I can't build without threads, it'd be extremely hard to tell it that it can remove all threading related things (like TLS) and inline them. This means I'll have to patch CPython on my own to remove thread support, but unfortunately this also means I can't guarantee that it continues to work, as well as that it makes it a lot more likely that my build won't support many third-party libraries.

[brettcannon]

This has unfortunately turned out to be a blocker on getting WASI support as there's not direct threading support in WebAssembly.

[gpshead]

https://web.dev/webassembly-threads/ suggests otherwise.

[brettcannon]

https://web.dev/webassembly-threads/ suggests otherwise.

It actually doesn't as that post isn't for WASI, it's for WASM in the browser (I chose my acronyms carefully 😉).

WASI is like POSIX for WebAssembly, so it's meant for desktop usage and thus no browser or JS engine is involved to provide web workers.

[vstinner]

This has unfortunately turned out to be a blocker on getting WASI support as there's not direct threading support in WebAssembly.

This issue is now closed and unrelated to WASI support. Can you please open a new separated issue for it?

[serhiy-storchaka]

Concur with Victor. It is fine to add a link to other discussion in a closed issue, but an issue closed many years ago is not a good place for a new discussion at all.

[brettcannon]

I am not opening a new discussion; this is just recording this fact here as it has come up in other places on the internet.

[vstinner]

Either reopen the issue or open a new issue. Only people subscribed to this bug are aware of the recent comments. Closed issues are hidden from the bugs home page and from the default search.

If you consider that Python must again support thread-less platforms, IMO it's a new feature and a new issue must be opened.