Skip to content

fix: DaemonThreadPoolExecutor compat with Python 3.14 - #69311

Open
Aciredy wants to merge 1 commit into
NousResearch:mainfrom
Aciredy:fix/daemon-pool-py314-compat
Open

fix: DaemonThreadPoolExecutor compat with Python 3.14#69311
Aciredy wants to merge 1 commit into
NousResearch:mainfrom
Aciredy:fix/daemon-pool-py314-compat

Conversation

@Aciredy

@Aciredy Aciredy commented Jul 22, 2026

Copy link
Copy Markdown

Problem

Python 3.14 removed _initializer / _initargs from ThreadPoolExecutor.__init__, replacing them with prepare_context()_create_worker_context / _resolve_work_item_task. The _worker function signature also changed from 4 args to 3.

DaemonThreadPoolExecutor._adjust_thread_count() reads self._initializer and self._initargs unconditionally, so on Python 3.14 every submit() call raises:

AttributeError: 'DaemonThreadPoolExecutor' object has no attribute '_initializer'

This breaks all concurrent tool execution (web_search, web_extract, session_search, search_files, etc.) when the agent runs under Python 3.14.

Fix

  • Added explicit __init__ that calls super().__init__() and backfills _initializer/_initargs if the stdlib didn't set them (3.14 safety net for any external code reading those attributes).
  • _adjust_thread_count() now branches on sys.version_info >= (3, 14):
    • 3.14+: _worker(executor_ref, self._create_worker_context(), self._work_queue)
    • 3.8–3.13: _worker(executor_ref, self._work_queue, self._initializer, self._initargs)

Tested

  • Python 3.11.14 (venv gateway) ✅
  • Python 3.14.4 (Homebrew WebUI) ✅
  • Both with and without initializer/initargs arguments

Python 3.14 removed _initializer/_initargs from ThreadPoolExecutor.__init__
(replaced by prepare_context() -> _create_worker_context). Also changed
_worker signature from 4 args (executor_ref, work_queue, initializer,
initargs) to 3 args (executor_ref, ctx, work_queue).

DaemonThreadPoolExecutor._adjust_thread_count() read the old attributes
unconditionally, causing AttributeError on every concurrent tool batch
under Python 3.14.

Fix: version-branched _adjust_thread_count() + explicit __init__ that
backfills _initializer/_initargs for any external code reading them.
Tested on Python 3.11.14 and 3.14.4.
@alt-glitch alt-glitch added type/bug Something isn't working comp/tools Tool registry, model_tools, toolsets P3 Low — cosmetic, nice to have duplicate This issue or pull request already exists sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades labels Jul 22, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.

Duplicate of open #65182: both repair the same tools/daemon_pool.py WorkerContext/_worker contract for CPython 3.14. The initializer backfill here is not a materially distinct fix; #57459 remains related but lacks the complete WorkerContext repair.

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing a real compatibility break: current main still passes removed initializer attributes in tools/daemon_pool.py:55-60, while CPython 3.14 requires _worker(executor_ref, ctx, work_queue) (Lib/concurrent/futures/thread.py:97-105,233-236). The proposed WorkerContext tuple at PR lines 77-81 is the correct repair direction.

Problems

  • The PR changes only tools/daemon_pool.py; it adds no regression coverage. Existing tests/tools/test_daemon_pool.py:20-83 does not explicitly assert the 3.14 WorkerContext tuple.
  • Proposed line 44 selects the tuple solely by sys.version_info. This code relies on a private stdlib interface, so select the branch from the actual capability (_create_worker_context) instead of a version assumption.

Suggested changes

  • Add a mocked worker-spawn test that captures threading.Thread arguments and verifies the three-element WorkerContext tuple and daemon=True without requiring a 3.14 CI interpreter.
  • Prefer hasattr(self, "_create_worker_context") for the branch condition, as in related open #65182.
  • The _initializer/_initargs backfill is not used by the proposed 3.14 branch; remove it unless an identified consumer needs it.

Automated hermes-sweeper review.

Comment thread tools/daemon_pool.py
# to
# _worker(executor_ref, ctx, work_queue) # 3.14+
# The override below must branch on the running interpreter version.
_PY314 = sys.version_info >= (3, 14)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please gate this on the interface actually required, e.g. hasattr(self, "_create_worker_context"), rather than sys.version_info. This override depends on CPython private executor internals; the presence of that factory directly determines whether the three-argument WorkerContext tuple is valid.

Comment thread tools/daemon_pool.py
class DaemonThreadPoolExecutor(ThreadPoolExecutor):
"""ThreadPoolExecutor variant whose workers do not block process exit."""

def __init__(self, *args, **kwargs):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new 3.14 branch does not read either backfilled attribute, and the in-tree daemon-pool users only construct or submit through this executor. Please remove this compatibility backfill unless there is a concrete external consumer to preserve; it is unrelated to the WorkerContext repair.

@GottZ

GottZ commented Aug 3, 2026

Copy link
Copy Markdown

This was generated by AI during triage.

Summary

Eighteen PRs address or reference the Python 3.14 DaemonThreadPoolExecutor failure: sixteen visible diffs construct the required three-element WorkerContext tuple with varying detection and test strategies, while #59157 and #63777 retain the incompatible four-argument worker call. #61224 overlaps the executor repair but also contains distinct gateway-liveness and dotenv-reload changes.

Related pull requests

Duplicates

#58598, #58699, #59897, #60061, #63780, #65182, #69108, #69209, #69311, #72955, #74452, #76212, #76756, and #76817 substantially implement the same WorkerContext-aware repair, while #57459 now overlaps that family and #59157/#63777 duplicate each other as incomplete four-argument fallbacks. #61224 overlaps only in tools/daemon_pool.py and retains two distinct fixes.

Suggested consolidation

Keep #58699 open with a salvage path: author action should rebase it onto main while preserving its signature/capability-based dispatch and mocked WorkerContext coverage; also keep recorded best fix #65182 open long enough to add or adopt #76756's mocked tuple test and resolve consolidation between the two recorded best-fix branches. Close #69311 as a duplicate of #65182 despite its keep_open review because its diff uses the less robust version gate, adds unrelated backfills, and lacks the requested test; likewise close #57459, #60061, #69108, #69209, #74452, and #76817 as duplicates of #58699 or #65182 despite their keep_open reviews because their visible executor diffs add no stronger cause coverage, keep already-closed duplicates closed, and request that #61224 rebase or split out its gateway-liveness and dotenv-retry changes.

Complex graph

flowchart TD
    classDef open fill:#dbeafe,stroke:#1d4ed8,color:#1e3a8a
    classDef merged fill:#dcfce7,stroke:#15803d,color:#14532d
    classDef closed fill:#e5e7eb,stroke:#6b7280,color:#1f2937
    classDef unverified fill:#f3f4f6,stroke:#9ca3af,color:#374151
    classDef best stroke-width:3px,stroke:#b45309
    classDef target stroke-width:3px,stroke:#4338ca
    I58596(["issue #58596 (open)"])
    I59896(["issue #59896 (open)"])
    I63769(["issue #63769 (open)"])
    I69359(["issue #69359 (open)"])
    I76621(["issue #76621 (open)"])
    subgraph Dup57459 ["PRs duplicating each other"]
        P57459["PR #57459 (open)"]
        P58598["PR #58598 (closed)"]
        P58699["PR #58699 (open)"]
        P59157["PR #59157 (closed)"]
        P59897["PR #59897 (closed)"]
        P60061["PR #60061 (open)"]
        P61224["PR #61224 (open)"]
        P63777["PR #63777 (closed)"]
        P63780["PR #63780 (closed)"]
        P65182["PR #65182 (open)"]
        P69108["PR #69108 (open)"]
        P69209["PR #69209 (open)"]
        P69311["PR #69311 (open)"]
        P72955["PR #72955 (closed)"]
        P74452["PR #74452 (open)"]
        P76212["PR #76212 (closed)"]
        P76756["PR #76756 (closed)"]
        P76817["PR #76817 (open)"]
    end
    P69311 -->|fixes| I58596
    P69311 -->|fixes| I59896
    P69311 -->|fixes| I63769
    P69311 -->|fixes| I69359
    P69311 -->|fixes| I76621
    class I58596 open
    class I59896 open
    class I63769 open
    class I69359 open
    class I76621 open
    class P57459 open
    class P58598 closed
    class P58699 open
    class P59157 closed
    class P59897 closed
    class P60061 open
    class P61224 open
    class P63777 closed
    class P63780 closed
    class P65182 open
    class P69108 open
    class P69209 open
    class P69311 open
    class P72955 closed
    class P74452 open
    class P76212 closed
    class P76756 closed
    class P76817 open
    class P58699 best
    class P58699 best
    class P58699 best
    class P58699 best
    class P58699 best
    class P65182 best
    class P69311 target
    click I58596 "https://github.com/NousResearch/hermes-agent/issues/58596"
    click I59896 "https://github.com/NousResearch/hermes-agent/issues/59896"
    click I63769 "https://github.com/NousResearch/hermes-agent/issues/63769"
    click I69359 "https://github.com/NousResearch/hermes-agent/issues/69359"
    click I76621 "https://github.com/NousResearch/hermes-agent/issues/76621"
    click P57459 "https://github.com/NousResearch/hermes-agent/pull/57459"
    click P58598 "https://github.com/NousResearch/hermes-agent/pull/58598"
    click P58699 "https://github.com/NousResearch/hermes-agent/pull/58699"
    click P59157 "https://github.com/NousResearch/hermes-agent/pull/59157"
    click P59897 "https://github.com/NousResearch/hermes-agent/pull/59897"
    click P60061 "https://github.com/NousResearch/hermes-agent/pull/60061"
    click P61224 "https://github.com/NousResearch/hermes-agent/pull/61224"
    click P63777 "https://github.com/NousResearch/hermes-agent/pull/63777"
    click P63780 "https://github.com/NousResearch/hermes-agent/pull/63780"
    click P65182 "https://github.com/NousResearch/hermes-agent/pull/65182"
    click P69108 "https://github.com/NousResearch/hermes-agent/pull/69108"
    click P69209 "https://github.com/NousResearch/hermes-agent/pull/69209"
    click P69311 "https://github.com/NousResearch/hermes-agent/pull/69311"
    click P72955 "https://github.com/NousResearch/hermes-agent/pull/72955"
    click P74452 "https://github.com/NousResearch/hermes-agent/pull/74452"
    click P76212 "https://github.com/NousResearch/hermes-agent/pull/76212"
    click P76756 "https://github.com/NousResearch/hermes-agent/pull/76756"
    click P76817 "https://github.com/NousResearch/hermes-agent/pull/76817"
Loading

Graph: solid arrow = fixes / best fix, dashed arrow = partial or unverified (see edge label); boxed group = PRs duplicating each other; amber border = best fix; indigo border = target; gray node = closed (state tag in the node label).

Cross-PR triage: Reviewed 18 pull requests and 5 issues in this complex. Each diff was read against this issue; Assessment working set: 70 kB of PR diffs, 52 kB of issue/PR text, 40 kB of discussion (59 comments), 115 verify verdicts. verdicts reflect diff content, not PR titles. Part of an automated triage batch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/tools Tool registry, model_tools, toolsets duplicate This issue or pull request already exists P3 Low — cosmetic, nice to have sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants