Skip to content

fix(tools): support CPython 3.14 ThreadPoolExecutor internals in DaemonThreadPoolExecutor - #69209

Open
Soju06 wants to merge 1 commit into
NousResearch:mainfrom
Soju06:upstream-pr/daemon-pool-py314
Open

fix(tools): support CPython 3.14 ThreadPoolExecutor internals in DaemonThreadPoolExecutor#69209
Soju06 wants to merge 1 commit into
NousResearch:mainfrom
Soju06:upstream-pr/daemon-pool-py314

Conversation

@Soju06

@Soju06 Soju06 commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

Problem

DaemonThreadPoolExecutor._adjust_thread_count mirrors CPython 3.8–3.13 ThreadPoolExecutor internals (self._initializer / self._initargs and the 5-arg _worker signature). CPython 3.14 moved per-worker state into a WorkerContext built by the new prepare_context() classmethod and changed _worker to (executor_ref, ctx, work_queue).

Under Python 3.14 every submit() dies on first dispatch:

AttributeError: 'DaemonThreadPoolExecutor' object has no attribute '_initializer'

This breaks all concurrent tool batches and background memory sync. Reproduces on pure main: tests/tools/test_daemon_pool.py, tests/agent/test_memory_async_sync.py, and the TestSegmentedDispatchIntegration cluster all fail under a 3.14 interpreter.

Fix

Branch on hasattr(ThreadPoolExecutor, "prepare_context") and pass the matching worker args on each interpreter family, keeping daemon=True and the deliberately-skipped _threads_queues registration on both.

Testing

  • Python 3.14.6: tests/tools/test_daemon_pool.py, tests/agent/test_memory_async_sync.py, tests/run_agent/test_tool_batch_segmentation.py — 38 passed, 1 skipped (all failed before on 3.14).
  • The 3.8–3.13 path is untouched (same attributes/signature as before, now behind the feature check).

🤖 Generated with Claude Code

…monThreadPoolExecutor

CPython 3.14 moved per-worker state into a WorkerContext built by the new
prepare_context() classmethod and changed _worker's signature to
(executor_ref, ctx, work_queue). DaemonThreadPoolExecutor mirrored the
3.8–3.13 _adjust_thread_count and crashed with
AttributeError: '_initializer' on first submit under the project venv
(now CPython 3.14.6) — breaking every background memory-sync dispatch and
this patch's WAL/mirror/ingest-gate tests. Pre-existing at the v2026.7.20
tag: upstream's own tests/tools/test_daemon_pool.py and
tests/agent/test_memory_async_sync.py fail there under 3.14
(tools/daemon_pool.py last touched by 3f2a56d; upstream declares
3.11–3.13 support in d84a2af).

Branch on hasattr(ThreadPoolExecutor, "prepare_context") and pass the
matching worker args, keeping daemon=True and the skipped _threads_queues
registration on both interpreter families. Standalone stack-root patch: needed by
every branch whose tests touch the sync executor; upstream PR
patch + upstream PR.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Soju06 added a commit to Soju06/hermes-agent that referenced this pull request Jul 22, 2026
…tream (PR NousResearch#69209)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@alt-glitch alt-glitch added type/bug Something isn't working P3 Low — cosmetic, nice to have comp/tools Tool registry, model_tools, toolsets duplicate This issue or pull request already exists labels Jul 22, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.

Duplicate of #65182: both live diffs implement the same CPython 3.14 WorkerContext and worker-argument compatibility repair in tools/daemon_pool.py.

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for targeting the actual CPython 3.14 contract change. Current main still passes the legacy tuple through tools/daemon_pool.py:55-60, while CPython 3.14 invokes _worker with (executor_ref, self._create_worker_context(), work_queue) (Lib/concurrent/futures/thread.py:97, 185-187, 233-236). The new branch is therefore the correct repair direction.

Problems

  • The diff adds no regression coverage. The supported interpreter range remains <3.14 in pyproject.toml:15, and tests/tools/test_daemon_pool.py:20-83 does not exercise or mock the WorkerContext branch.
  • The member triage comment on this PR links open #65182 as the same WorkerContext repair; its live diff implements the same three-argument adaptation.

Suggested changes

  • Add a mocked threading.Thread construction test that verifies the WorkerContext tuple and daemon=True without needing CPython 3.14 in CI.
  • Coordinate consolidation with #65182 so the selected repair carries regression coverage.

Automated hermes-sweeper review.

Comment thread tools/daemon_pool.py
self._create_worker_context(),
self._work_queue,
)
else:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a mocked regression test for this branch. CI currently supports Python <3.14, so the normal daemon-pool submission tests cannot execute this WorkerContext tuple.

@GottZ

GottZ commented Aug 3, 2026

Copy link
Copy Markdown

This was generated by AI during triage.

Summary

Eighteen PRs address or reference the shared Python 3.14 DaemonThreadPoolExecutor failure: sixteen visible diffs construct the required three-element WorkerContext tuple with differing detection and test coverage, while #59157 and #63777 retain the incompatible four-argument call. #61224 additionally contains distinct gateway-liveness and dotenv-reload fixes.

Related pull requests

Duplicates

#57459, #58598, #58699, #59897, #60061, #63780, #65182, #69108, #69209, #69311, #72955, #74452, #76212, #76756, and #76817 substantially implement the same WorkerContext-aware spawn repair; #59157 and #63777 are duplicate incomplete four-argument fallbacks. #61224 overlaps only in its daemon-pool portion because its gateway and env-loader changes are distinct.

Suggested consolidation

Author action: rebase #58699 onto main, or split out the part that can merge, preserving its recorded best-fix implementation and mocked target-path coverage; keep recorded best fix #65182 open with the salvage path of adopting the focused mocked test and reconciling its implementation with #58699. Despite the keep_open reviews or verdicts on #57459, #60061, #69108, #69209, #69311, #74452, and #76817, their visible diffs add no stronger cause coverage than #58699/#65182, so close them as explicit duplicates; keep the already-closed PRs closed, and keep #61224 open only with a salvage path that splits its distinct gateway-liveness and dotenv-retry work from the duplicate daemon-pool repair.

Complex graph

flowchart TD
    classDef open fill:#dbeafe,stroke:#1d4ed8,color:#1e3a8a
    classDef merged fill:#dcfce7,stroke:#15803d,color:#14532d
    classDef closed fill:#e5e7eb,stroke:#6b7280,color:#1f2937
    classDef unverified fill:#f3f4f6,stroke:#9ca3af,color:#374151
    classDef best stroke-width:3px,stroke:#b45309
    classDef target stroke-width:3px,stroke:#4338ca
    I58596(["issue #58596 (open)"])
    I59896(["issue #59896 (open)"])
    I63769(["issue #63769 (open)"])
    I69359(["issue #69359 (open)"])
    I76621(["issue #76621 (open)"])
    subgraph Dup57459 ["PRs duplicating each other"]
        P57459["PR #57459 (open)"]
        P58598["PR #58598 (closed)"]
        P58699["PR #58699 (open)"]
        P59157["PR #59157 (closed)"]
        P59897["PR #59897 (closed)"]
        P60061["PR #60061 (open)"]
        P61224["PR #61224 (open)"]
        P63777["PR #63777 (closed)"]
        P63780["PR #63780 (closed)"]
        P65182["PR #65182 (open)"]
        P69108["PR #69108 (open)"]
        P69209["PR #69209 (open)"]
        P69311["PR #69311 (open)"]
        P72955["PR #72955 (closed)"]
        P74452["PR #74452 (open)"]
        P76212["PR #76212 (closed)"]
        P76756["PR #76756 (closed)"]
        P76817["PR #76817 (open)"]
    end
    P69209 -->|fixes| I58596
    P69209 -->|fixes| I59896
    P69209 -->|fixes| I63769
    P69209 -->|fixes| I69359
    P69209 -->|fixes| I76621
    class I58596 open
    class I59896 open
    class I63769 open
    class I69359 open
    class I76621 open
    class P57459 open
    class P58598 closed
    class P58699 open
    class P59157 closed
    class P59897 closed
    class P60061 open
    class P61224 open
    class P63777 closed
    class P63780 closed
    class P65182 open
    class P69108 open
    class P69209 open
    class P69311 open
    class P72955 closed
    class P74452 open
    class P76212 closed
    class P76756 closed
    class P76817 open
    class P58699 best
    class P58699 best
    class P58699 best
    class P58699 best
    class P58699 best
    class P65182 best
    class P69209 target
    click I58596 "https://github.com/NousResearch/hermes-agent/issues/58596"
    click I59896 "https://github.com/NousResearch/hermes-agent/issues/59896"
    click I63769 "https://github.com/NousResearch/hermes-agent/issues/63769"
    click I69359 "https://github.com/NousResearch/hermes-agent/issues/69359"
    click I76621 "https://github.com/NousResearch/hermes-agent/issues/76621"
    click P57459 "https://github.com/NousResearch/hermes-agent/pull/57459"
    click P58598 "https://github.com/NousResearch/hermes-agent/pull/58598"
    click P58699 "https://github.com/NousResearch/hermes-agent/pull/58699"
    click P59157 "https://github.com/NousResearch/hermes-agent/pull/59157"
    click P59897 "https://github.com/NousResearch/hermes-agent/pull/59897"
    click P60061 "https://github.com/NousResearch/hermes-agent/pull/60061"
    click P61224 "https://github.com/NousResearch/hermes-agent/pull/61224"
    click P63777 "https://github.com/NousResearch/hermes-agent/pull/63777"
    click P63780 "https://github.com/NousResearch/hermes-agent/pull/63780"
    click P65182 "https://github.com/NousResearch/hermes-agent/pull/65182"
    click P69108 "https://github.com/NousResearch/hermes-agent/pull/69108"
    click P69209 "https://github.com/NousResearch/hermes-agent/pull/69209"
    click P69311 "https://github.com/NousResearch/hermes-agent/pull/69311"
    click P72955 "https://github.com/NousResearch/hermes-agent/pull/72955"
    click P74452 "https://github.com/NousResearch/hermes-agent/pull/74452"
    click P76212 "https://github.com/NousResearch/hermes-agent/pull/76212"
    click P76756 "https://github.com/NousResearch/hermes-agent/pull/76756"
    click P76817 "https://github.com/NousResearch/hermes-agent/pull/76817"
Loading

Graph: solid arrow = fixes / best fix, dashed arrow = partial or unverified (see edge label); boxed group = PRs duplicating each other; amber border = best fix; indigo border = target; gray node = closed (state tag in the node label).

Cross-PR triage: Reviewed 18 pull requests and 5 issues in this complex. Each diff was read against this issue; Assessment working set: 70 kB of PR diffs, 52 kB of issue/PR text, 40 kB of discussion (59 comments), 115 verify verdicts. verdicts reflect diff content, not PR titles. Part of an automated triage batch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/tools Tool registry, model_tools, toolsets duplicate This issue or pull request already exists P3 Low — cosmetic, nice to have sweeper:blast-broad Sweeper blast radius: broad — a core path most sessions hit sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants