fix(tools): spawn daemon workers with the 3.14 worker context - #72955
Closed
mrwogu wants to merge 1 commit into
Closed
fix(tools): spawn daemon workers with the 3.14 worker context#72955mrwogu wants to merge 1 commit into
mrwogu wants to merge 1 commit into
Conversation
CPython 3.14 replaced the `(work_queue, initializer, initargs)` worker arguments with a per-worker context object and dropped the executor's `_initializer` / `_initargs` attributes. `_adjust_thread_count` mirrored the 3.8-3.13 shape, so every parallel tool batch raised `AttributeError: 'DaemonThreadPoolExecutor' object has no attribute '_initializer'` in `submit()`. Worker arguments are now built from `_create_worker_context()` when the interpreter provides it and from the legacy attributes otherwise, so the same source works on 3.11 through 3.14. A regression test ties the argument count to the stdlib `_worker` signature so the next CPython change fails loudly instead of at runtime.
Collaborator
Author
|
Agreed on the duplicate call: #65182 is older and uses the same The parts of this PR that are not in #65182 are moved to #65182 (comment):
|
This was referenced Jul 28, 2026
Open
Open
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Fixes the
DaemonThreadPoolExecutorcrash on Python 3.14 and, unlike the open patch in #57459, keeps the pool actually working there.CPython 3.14 changed
concurrent.futures.thread._workerfrom(executor_ref, work_queue, initializer, initargs)to(executor_ref, ctx, work_queue), wherectxcomes fromself._create_worker_context().ThreadPoolExecutor.__init__no longer sets_initializer/_initargsat all.tools/daemon_pool.pystill mirrored the 3.8-3.13 shape, so every worker spawn raised:Live traceback from a Homebrew install running on 3.14.6 (
hermes-agent/2026.6.5), triggered by an ordinary parallel tool batch:The agent surfaced it as
Error during OpenAI-compatible API call #N, which sends the outer loop into retries for what is really a local spawn failure.Why this approach rather than #57459
#57459 reads the two attributes through
getattr(..., None)but still passes a 4-tuple to a 3-parameter_worker. That silences theAttributeErroratsubmit()and moves the failure into the worker thread, whereTypeErrorkills the thread and the future never resolves. Loud crash becomes a hang. Probe on this machine, same class shapes side by side:So the fix has to switch the whole argument shape, not just guard the attribute reads. Same code site as #57459, #58598, #59157, #63777 and #63780; this one is the version that survives a
submit()on 3.14.requires-pythonis>=3.11,<3.14, which is why #50077 was closed as unreproducible. That bound does not protect installs in practice: the Homebrew formula builds the app againstpython@3.14, so the interpreter that runshermesis 3.14.6 regardless. The patch keeps the legacy branch intact, so nothing changes on 3.11-3.13 whether or not the supported range moves later.Related Issue
Fixes #58596
Type of Change
Changes Made
tools/daemon_pool.py:_adjust_thread_countnow takes its thread arguments from a new_worker_args()helper, which returns(executor_ref, create_context(), work_queue)when the interpreter exposes_create_worker_contextand the legacy(executor_ref, work_queue, initializer, initargs)tuple otherwise. Daemon threads and the skipped_threads_queuesregistration are unchanged, so abandoned workers still cannot block interpreter exit.tests/tools/test_daemon_pool.py: addstest_worker_args_match_stdlib_worker_signature, which asserts the argument count matchesinspect.signature(_worker)on whatever interpreter runs the suite. The next CPython signature change then fails in CI instead of in a user's parallel tool batch.How to Test
python3.14 -c "import sys; sys.path.insert(0,'.'); from tools.daemon_pool import DaemonThreadPoolExecutor as P; p=P(max_workers=1); print(p.submit(lambda: 1+1).result(timeout=5))"raisesAttributeError: ... has no attribute '_initializer'.2.python3.14withmax_workers=2,initializer/initargs, then assert the worker reportsthreading.current_thread().daemon is True, that it is absent fromconcurrent.futures.thread._threads_queues, that the initializer ran, and that a second submit reuses the same thread id. All four hold.scripts/run_tests.sh tests/tools/test_daemon_pool.py -q(5 passed) and the consumer slicesscripts/run_tests.sh tests/tools/test_daemon_pool.py tests/tools/test_async_delegation.py tests/tools/test_delegate.py -q(205 passed).python3 scripts/check-windows-footguns.py tools/daemon_pool.py tests/tools/test_daemon_pool.pyreports no findings.Checklist
Code
fix(scope):,feat(scope):, etc.)pytest tests/ -qand all tests pass — viascripts/run_tests.shon the touched module and its consumersDocumentation & Housekeeping
docs/, docstrings) — module docstring already documents the mirrored implementation; the version range comment is updated to 3.8-3.14Screenshots / Logs
Probe script used for the comparison above (throwaway, not part of the diff): it subclasses
ThreadPoolExecutortwice, once with the #57459 argument shape and once with this PR's, then submits one task withresult(timeout=5)on each. Output for 3.14.6 and 3.11.15 is quoted in full in the section above.