Skip to content

fix(daemon_pool): support Python 3.14 ThreadPoolExecutor WorkerContext signature - #63780

Closed
xxiaoxiong wants to merge 1 commit into
NousResearch:mainfrom
xxiaoxiong:fix/63769-daemon-pool-python-314-compat
Closed

fix(daemon_pool): support Python 3.14 ThreadPoolExecutor WorkerContext signature#63780
xxiaoxiong wants to merge 1 commit into
NousResearch:mainfrom
xxiaoxiong:fix/63769-daemon-pool-python-314-compat

Conversation

@xxiaoxiong

Copy link
Copy Markdown

Problem

Python 3.14 refactored concurrent.futures.ThreadPoolExecutor: the _worker target now takes (executor_ref, ctx, work_queue) where ctx is a WorkerContext produced by self._create_worker_context(), replacing the legacy (executor_ref, work_queue, initializer, initargs) signature. __init__ also no longer sets self._initializer / self._initargs.

DaemonThreadPoolExecutor._adjust_thread_count still passed the legacy 4-arg shape, so on Python 3.14 every spawn raised:

AttributeError: 'DaemonThreadPoolExecutor' object has no attribute '_initializer'

This breaks every daemon-pool consumer on 3.14+:

  • tools/delegate_tool.py (subagent batch execution + timeout wrappers)
  • tools/async_delegation.py (background delegate_task dispatch)
  • tools/skills_hub.py (catalog fan-out)

When the daemon pool cannot spawn, delegate_task's batch path collapses to a synchronous inline run, destroying parallelism and silently wedging subagent fan-out.

Fixes #63769.

Fix

Detect the new layout once at import time by inspecting _worker's signature (param count 3 = 3.14+, 4 = 3.8–3.13). The _worker signature is the load-bearing contract: stable across patch releases and immune to the instance-vs-class attribute pitfall that breaks hasattr(ThreadPoolExecutor, '_initializer') (3.11 only sets it on instances, so the class check is False on every version).

The 3.14+ branch passes self._create_worker_context() as ctx; the 3.8–3.13 branch is unchanged. Daemon behavior (daemon=True, no _threads_queues registration) is preserved on both paths.

Testing

tests/tools/test_daemon_pool.py:

  • Existing 4 tests still pass on 3.8–3.13 (unchanged behavior)
  • New: test_python_314_worker_context_signature_does_not_crash — guards the regression: submit must not raise AttributeError, worker must be daemon, must not be in _threads_queues, initializer must run via WorkerContext. Skips on 3.11–3.13.
  • New: test_python_314_submit_many_tasks_exercises_reused_workers — guards against worker-context-tied-to-first-submit regression on the idle-semaphore reuse path. Runs on all versions.

Verified on both runtimes

Python daemon_pool tests delegate + async_delegation tests
3.11.15 5 passed, 1 skipped 27 passed (no regressions)
3.14.4 6 passed (regression test now executes)

Notes

  • The detector uses _worker's function signatur e via inspect.signature, not class-level attribute probes — this is the only reliable cross-version check.
  • _PY314_PLUS is a module-level constant computed once at import time, so the hot path (_adjust_thread_count) stays branch-lite.
  • Test helper _supports_worker_context() re-uses _PY314_PLUS so the gate and dispatch logic stay in lockstep.

…t signature

Python 3.14 refactored concurrent.futures.ThreadPoolExecutor: the
`_worker` target now takes `(executor_ref, ctx, work_queue)` where `ctx`
is a WorkerContext produced by `self._create_worker_context()`, replacing
the legacy `(executor_ref, work_queue, initializer, initargs)` signature.
`__init__` also no longer sets `self._initializer` / `self._initargs`.

`DaemonThreadPoolExecutor._adjust_thread_count` still passed the legacy
4-arg shape, so on 3.14 every spawn raised:
  AttributeError: 'DaemonThreadPoolExecutor' object has no attribute '_initializer'

This breaks every daemon-pool consumer on 3.14+:
  - tools/delegate_tool.py (subagent batch execution + timeout wrappers)
  - tools/async_delegation.py (background delegate_task dispatch)
  - tools/skills_hub.py (catalog fan-out)
When the daemon pool cannot spawn, delegate_task's batch path collapses
to a synchronous inline run, destroying the parallelism the user asked
for and silently wedging subagent fan-out.

Fix
---
Detect the new layout once at import time by inspecting `_worker`'s
signature (param count 3 = 3.14+, 4 = 3.8–3.13). The `_worker` signature
is the load-bearing contract: it's stable across patch releases and
immune to the instance-vs-class attribute pitfall that breaks
`hasattr(ThreadPoolExecutor, '_initializer')` (3.11 only sets it on
instances, so the class check is False on every version).

The 3.14+ branch passes `self._create_worker_context()` as `ctx`; the
3.8–3.13 branch is unchanged. Daemon behavior (daemon=True, no
_threads_queues registration) is preserved on both paths.

Tests
-----
 tests/tools/test_daemon_pool.py:
  - existing 4 tests still pass on 3.8–3.13 (unchanged behavior).
  - new test_python_314_worker_context_signature_does_not_crash guards
    the regression: submit must not raise AttributeError, the worker
    must be daemon, must not be in `_threads_queues`, and the
    initializer must run via the WorkerContext path. Skips on 3.11–3.13.
  - new test_python_314_submit_many_tasks_exercises_reused_workers
    guards against a worker-context-tied-to-first-submit regression on
    the idle-semaphore reuse path. Runs on all versions.

Verified
--------
  Python 3.11.15: 5 passed, 1 skipped
  Python 3.14.4:   6 passed  (regression test now executes)
  Python 3.11.15:  27 passed in tests/{test_delegate_cascade_49148,
                  tools/test_async_delegation}.py — no regressions to
                  delegate_task / async delegation paths.

Closes NousResearch#63769.
@alt-glitch alt-glitch added type/bug Something isn't working comp/tools Tool registry, model_tools, toolsets P3 Low — cosmetic, nice to have duplicate This issue or pull request already exists labels Jul 13, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.

Duplicate of #57459 (earliest open) — same tools/daemon_pool.py _adjust_thread_count site and same Python 3.14 _worker-signature crash. Mechanism differs only in the detection probe (inspect.signature here vs getattr fallback in #57459). Issue #58596 is the bug spec; #47634/#50077 patch the distinct async_delegation.py site (related).

@xxiaoxiong

Copy link
Copy Markdown
Author

Closing — Duplicate of #57459 (Python 3.14 daemon_pool compat, same site/issue, different probe mechanism)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/tools Tool registry, model_tools, toolsets duplicate This issue or pull request already exists P3 Low — cosmetic, nice to have type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(delegation): saturated child pool crashes synchronous fallback with missing _initializer

2 participants