fix: DaemonThreadPoolExecutor compat with Python 3.14 - #69311
Conversation
Python 3.14 removed _initializer/_initargs from ThreadPoolExecutor.__init__ (replaced by prepare_context() -> _create_worker_context). Also changed _worker signature from 4 args (executor_ref, work_queue, initializer, initargs) to 3 args (executor_ref, ctx, work_queue). DaemonThreadPoolExecutor._adjust_thread_count() read the old attributes unconditionally, causing AttributeError on every concurrent tool batch under Python 3.14. Fix: version-branched _adjust_thread_count() + explicit __init__ that backfills _initializer/_initargs for any external code reading them. Tested on Python 3.11.14 and 3.14.4.
teknium1
left a comment
There was a problem hiding this comment.
Thanks for addressing a real compatibility break: current main still passes removed initializer attributes in tools/daemon_pool.py:55-60, while CPython 3.14 requires _worker(executor_ref, ctx, work_queue) (Lib/concurrent/futures/thread.py:97-105,233-236). The proposed WorkerContext tuple at PR lines 77-81 is the correct repair direction.
Problems
- The PR changes only
tools/daemon_pool.py; it adds no regression coverage. Existingtests/tools/test_daemon_pool.py:20-83does not explicitly assert the 3.14 WorkerContext tuple. - Proposed line 44 selects the tuple solely by
sys.version_info. This code relies on a private stdlib interface, so select the branch from the actual capability (_create_worker_context) instead of a version assumption.
Suggested changes
- Add a mocked worker-spawn test that captures
threading.Threadarguments and verifies the three-element WorkerContext tuple anddaemon=Truewithout requiring a 3.14 CI interpreter. - Prefer
hasattr(self, "_create_worker_context")for the branch condition, as in related open #65182. - The
_initializer/_initargsbackfill is not used by the proposed 3.14 branch; remove it unless an identified consumer needs it.
Automated hermes-sweeper review.
| # to | ||
| # _worker(executor_ref, ctx, work_queue) # 3.14+ | ||
| # The override below must branch on the running interpreter version. | ||
| _PY314 = sys.version_info >= (3, 14) |
There was a problem hiding this comment.
Please gate this on the interface actually required, e.g. hasattr(self, "_create_worker_context"), rather than sys.version_info. This override depends on CPython private executor internals; the presence of that factory directly determines whether the three-argument WorkerContext tuple is valid.
| class DaemonThreadPoolExecutor(ThreadPoolExecutor): | ||
| """ThreadPoolExecutor variant whose workers do not block process exit.""" | ||
|
|
||
| def __init__(self, *args, **kwargs): |
There was a problem hiding this comment.
The new 3.14 branch does not read either backfilled attribute, and the in-tree daemon-pool users only construct or submit through this executor. Please remove this compatibility backfill unless there is a concrete external consumer to preserve; it is unrelated to the WorkerContext repair.
SummaryEighteen PRs address or reference the Python 3.14 Related pull requests
Duplicates#58598, #58699, #59897, #60061, #63780, #65182, #69108, #69209, #69311, #72955, #74452, #76212, #76756, and #76817 substantially implement the same WorkerContext-aware repair, while #57459 now overlaps that family and #59157/#63777 duplicate each other as incomplete four-argument fallbacks. #61224 overlaps only in Suggested consolidationKeep #58699 open with a salvage path: author action should rebase it onto main while preserving its signature/capability-based dispatch and mocked WorkerContext coverage; also keep recorded best fix #65182 open long enough to add or adopt #76756's mocked tuple test and resolve consolidation between the two recorded best-fix branches. Close #69311 as a duplicate of #65182 despite its keep_open review because its diff uses the less robust version gate, adds unrelated backfills, and lacks the requested test; likewise close #57459, #60061, #69108, #69209, #74452, and #76817 as duplicates of #58699 or #65182 despite their keep_open reviews because their visible executor diffs add no stronger cause coverage, keep already-closed duplicates closed, and request that #61224 rebase or split out its gateway-liveness and dotenv-retry changes. Complex graphflowchart TD
classDef open fill:#dbeafe,stroke:#1d4ed8,color:#1e3a8a
classDef merged fill:#dcfce7,stroke:#15803d,color:#14532d
classDef closed fill:#e5e7eb,stroke:#6b7280,color:#1f2937
classDef unverified fill:#f3f4f6,stroke:#9ca3af,color:#374151
classDef best stroke-width:3px,stroke:#b45309
classDef target stroke-width:3px,stroke:#4338ca
I58596(["issue #58596 (open)"])
I59896(["issue #59896 (open)"])
I63769(["issue #63769 (open)"])
I69359(["issue #69359 (open)"])
I76621(["issue #76621 (open)"])
subgraph Dup57459 ["PRs duplicating each other"]
P57459["PR #57459 (open)"]
P58598["PR #58598 (closed)"]
P58699["PR #58699 (open)"]
P59157["PR #59157 (closed)"]
P59897["PR #59897 (closed)"]
P60061["PR #60061 (open)"]
P61224["PR #61224 (open)"]
P63777["PR #63777 (closed)"]
P63780["PR #63780 (closed)"]
P65182["PR #65182 (open)"]
P69108["PR #69108 (open)"]
P69209["PR #69209 (open)"]
P69311["PR #69311 (open)"]
P72955["PR #72955 (closed)"]
P74452["PR #74452 (open)"]
P76212["PR #76212 (closed)"]
P76756["PR #76756 (closed)"]
P76817["PR #76817 (open)"]
end
P69311 -->|fixes| I58596
P69311 -->|fixes| I59896
P69311 -->|fixes| I63769
P69311 -->|fixes| I69359
P69311 -->|fixes| I76621
class I58596 open
class I59896 open
class I63769 open
class I69359 open
class I76621 open
class P57459 open
class P58598 closed
class P58699 open
class P59157 closed
class P59897 closed
class P60061 open
class P61224 open
class P63777 closed
class P63780 closed
class P65182 open
class P69108 open
class P69209 open
class P69311 open
class P72955 closed
class P74452 open
class P76212 closed
class P76756 closed
class P76817 open
class P58699 best
class P58699 best
class P58699 best
class P58699 best
class P58699 best
class P65182 best
class P69311 target
click I58596 "https://github.com/NousResearch/hermes-agent/issues/58596"
click I59896 "https://github.com/NousResearch/hermes-agent/issues/59896"
click I63769 "https://github.com/NousResearch/hermes-agent/issues/63769"
click I69359 "https://github.com/NousResearch/hermes-agent/issues/69359"
click I76621 "https://github.com/NousResearch/hermes-agent/issues/76621"
click P57459 "https://github.com/NousResearch/hermes-agent/pull/57459"
click P58598 "https://github.com/NousResearch/hermes-agent/pull/58598"
click P58699 "https://github.com/NousResearch/hermes-agent/pull/58699"
click P59157 "https://github.com/NousResearch/hermes-agent/pull/59157"
click P59897 "https://github.com/NousResearch/hermes-agent/pull/59897"
click P60061 "https://github.com/NousResearch/hermes-agent/pull/60061"
click P61224 "https://github.com/NousResearch/hermes-agent/pull/61224"
click P63777 "https://github.com/NousResearch/hermes-agent/pull/63777"
click P63780 "https://github.com/NousResearch/hermes-agent/pull/63780"
click P65182 "https://github.com/NousResearch/hermes-agent/pull/65182"
click P69108 "https://github.com/NousResearch/hermes-agent/pull/69108"
click P69209 "https://github.com/NousResearch/hermes-agent/pull/69209"
click P69311 "https://github.com/NousResearch/hermes-agent/pull/69311"
click P72955 "https://github.com/NousResearch/hermes-agent/pull/72955"
click P74452 "https://github.com/NousResearch/hermes-agent/pull/74452"
click P76212 "https://github.com/NousResearch/hermes-agent/pull/76212"
click P76756 "https://github.com/NousResearch/hermes-agent/pull/76756"
click P76817 "https://github.com/NousResearch/hermes-agent/pull/76817"
Graph: solid arrow = fixes / best fix, dashed arrow = partial or unverified (see edge label); boxed group = PRs duplicating each other; amber border = best fix; indigo border = target; gray node = closed (state tag in the node label). Cross-PR triage: Reviewed 18 pull requests and 5 issues in this complex. Each diff was read against this issue; Assessment working set: 70 kB of PR diffs, 52 kB of issue/PR text, 40 kB of discussion (59 comments), 115 verify verdicts. verdicts reflect diff content, not PR titles. Part of an automated triage batch. |
Problem
Python 3.14 removed
_initializer/_initargsfromThreadPoolExecutor.__init__, replacing them withprepare_context()→_create_worker_context/_resolve_work_item_task. The_workerfunction signature also changed from 4 args to 3.DaemonThreadPoolExecutor._adjust_thread_count()readsself._initializerandself._initargsunconditionally, so on Python 3.14 everysubmit()call raises:This breaks all concurrent tool execution (web_search, web_extract, session_search, search_files, etc.) when the agent runs under Python 3.14.
Fix
__init__that callssuper().__init__()and backfills_initializer/_initargsif the stdlib didn't set them (3.14 safety net for any external code reading those attributes)._adjust_thread_count()now branches onsys.version_info >= (3, 14):_worker(executor_ref, self._create_worker_context(), self._work_queue)_worker(executor_ref, self._work_queue, self._initializer, self._initargs)Tested
initializer/initargsarguments