fix(tools): support CPython 3.14 WorkerContext in DaemonThreadPoolExecutor - #53
Conversation
…cutor
CPython 3.14 replaced ThreadPoolExecutor's (initializer, initargs)
attribute pair + 4-arg _worker free function with a WorkerContext
object built via prepare_context()/_create_worker_context(). Our
DaemonThreadPoolExecutor._adjust_thread_count mirrored the pre-3.14
internals directly, so on 3.14 every worker spawn raised
AttributeError: 'DaemonThreadPoolExecutor' object has no attribute
'_initializer'
inside the worker thread, silently breaking every batch of 2+
concurrent tool calls (agent/tool_executor.py routes through this
pool). Reproduced empirically on the pre-fix code against the local
venv (Python 3.14.6, home = /opt/homebrew/opt/python@3.14):
$ python3 -c "from tools.daemon_pool import DaemonThreadPoolExecutor; \
DaemonThreadPoolExecutor(max_workers=2).submit(lambda: 1+1).result()"
AttributeError: 'DaemonThreadPoolExecutor' object has no attribute '_initializer'
CI runs Python 3.13 (pyproject.toml: requires-python = ">=3.11,<3.14"),
so this never surfaced there — only on interpreters that have since
moved to 3.14.
Fix: branch on sys.version_info at import time and mirror whichever
shape of _worker/_adjust_thread_count the running interpreter actually
has (confirmed against 3.14.6's concurrent.futures.thread source: the
3.14 _adjust_thread_count calls _worker(weakref, self._create_worker_context(),
work_queue), the legacy shape calls _worker(weakref, work_queue,
initializer, initargs)). Both branches keep the two behavioral changes
that make this pool useful: daemon=True and no _threads_queues
registration.
Added a regression test (test_many_concurrent_submits_like_tool_executor)
reproducing the tool-executor's concurrent-submit shape; verified it
fails with the exact AttributeError above against the pre-fix code and
passes against the fix, on both Python 3.11 (uv venv, officially
supported) and 3.14.6 (local dev venv).
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughThis PR updates daemon thread pool worker spawning to match CPython’s private ChangesDaemon Pool Worker Compatibility
Launchd Restart Test Determinism
Estimated code review effort: 4 (Complex) | ~35 minutes Sequence Diagram(s)sequenceDiagram
participant Executor as DaemonThreadPoolExecutor
participant Adjust as _adjust_thread_count
participant Spawn as _spawn_daemon_worker
participant Worker as _worker
Executor->>Adjust: request a new daemon thread
Adjust->>Spawn: _spawn_daemon_worker(self, thread_name, weakref_cb)
Spawn->>Worker: start thread with version-matched arguments
Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…a CI runners test_launchd_restart_boots_out_stale_registration_before_bootstrap called the real _launchd_domain() helper, which probes the live session type via launchctl and returns gui/<uid> on an Aqua GUI runner instead of the user/<uid> the test hardcodes into its plist path fixture. Pin the domain the same way 9c155f3 pinned it for the refresh test: stub _launchctl_session_managername to force headless, and make the launchctl print probe (used by _launchd_domain_for_existing_job) fail so domain resolution falls through to _launchd_domain() instead of matching an already-loaded gui/<uid> job. Unrelated to the daemon-pool py3.14 fix in this branch; found while tracking down PR #53's slice-4/8 CI failure.
…cutor (#53) * fix(tools): support CPython 3.14 WorkerContext in DaemonThreadPoolExecutor CPython 3.14 replaced ThreadPoolExecutor's (initializer, initargs) attribute pair + 4-arg _worker free function with a WorkerContext object built via prepare_context()/_create_worker_context(). Our DaemonThreadPoolExecutor._adjust_thread_count mirrored the pre-3.14 internals directly, so on 3.14 every worker spawn raised AttributeError: 'DaemonThreadPoolExecutor' object has no attribute '_initializer' inside the worker thread, silently breaking every batch of 2+ concurrent tool calls (agent/tool_executor.py routes through this pool). Reproduced empirically on the pre-fix code against the local venv (Python 3.14.6, home = /opt/homebrew/opt/python@3.14): $ python3 -c "from tools.daemon_pool import DaemonThreadPoolExecutor; \ DaemonThreadPoolExecutor(max_workers=2).submit(lambda: 1+1).result()" AttributeError: 'DaemonThreadPoolExecutor' object has no attribute '_initializer' CI runs Python 3.13 (pyproject.toml: requires-python = ">=3.11,<3.14"), so this never surfaced there — only on interpreters that have since moved to 3.14. Fix: branch on sys.version_info at import time and mirror whichever shape of _worker/_adjust_thread_count the running interpreter actually has (confirmed against 3.14.6's concurrent.futures.thread source: the 3.14 _adjust_thread_count calls _worker(weakref, self._create_worker_context(), work_queue), the legacy shape calls _worker(weakref, work_queue, initializer, initargs)). Both branches keep the two behavioral changes that make this pool useful: daemon=True and no _threads_queues registration. Added a regression test (test_many_concurrent_submits_like_tool_executor) reproducing the tool-executor's concurrent-submit shape; verified it fails with the exact AttributeError above against the pre-fix code and passes against the fix, on both Python 3.11 (uv venv, officially supported) and 3.14.6 (local dev venv). * fix(test): pin launchd domain in restart-recovery test to survive Aqua CI runners test_launchd_restart_boots_out_stale_registration_before_bootstrap called the real _launchd_domain() helper, which probes the live session type via launchctl and returns gui/<uid> on an Aqua GUI runner instead of the user/<uid> the test hardcodes into its plist path fixture. Pin the domain the same way 9c155f3 pinned it for the refresh test: stub _launchctl_session_managername to force headless, and make the launchctl print probe (used by _launchd_domain_for_existing_job) fail so domain resolution falls through to _launchd_domain() instead of matching an already-loaded gui/<uid> job. Unrelated to the daemon-pool py3.14 fix in this branch; found while tracking down PR #53's slice-4/8 CI failure. --------- Co-authored-by: Sahil (AI) <266772320+sahilm-ai@users.noreply.github.com>
…cutor (#53) * fix(tools): support CPython 3.14 WorkerContext in DaemonThreadPoolExecutor CPython 3.14 replaced ThreadPoolExecutor's (initializer, initargs) attribute pair + 4-arg _worker free function with a WorkerContext object built via prepare_context()/_create_worker_context(). Our DaemonThreadPoolExecutor._adjust_thread_count mirrored the pre-3.14 internals directly, so on 3.14 every worker spawn raised AttributeError: 'DaemonThreadPoolExecutor' object has no attribute '_initializer' inside the worker thread, silently breaking every batch of 2+ concurrent tool calls (agent/tool_executor.py routes through this pool). Reproduced empirically on the pre-fix code against the local venv (Python 3.14.6, home = /opt/homebrew/opt/python@3.14): $ python3 -c "from tools.daemon_pool import DaemonThreadPoolExecutor; \ DaemonThreadPoolExecutor(max_workers=2).submit(lambda: 1+1).result()" AttributeError: 'DaemonThreadPoolExecutor' object has no attribute '_initializer' CI runs Python 3.13 (pyproject.toml: requires-python = ">=3.11,<3.14"), so this never surfaced there — only on interpreters that have since moved to 3.14. Fix: branch on sys.version_info at import time and mirror whichever shape of _worker/_adjust_thread_count the running interpreter actually has (confirmed against 3.14.6's concurrent.futures.thread source: the 3.14 _adjust_thread_count calls _worker(weakref, self._create_worker_context(), work_queue), the legacy shape calls _worker(weakref, work_queue, initializer, initargs)). Both branches keep the two behavioral changes that make this pool useful: daemon=True and no _threads_queues registration. Added a regression test (test_many_concurrent_submits_like_tool_executor) reproducing the tool-executor's concurrent-submit shape; verified it fails with the exact AttributeError above against the pre-fix code and passes against the fix, on both Python 3.11 (uv venv, officially supported) and 3.14.6 (local dev venv). * fix(test): pin launchd domain in restart-recovery test to survive Aqua CI runners test_launchd_restart_boots_out_stale_registration_before_bootstrap called the real _launchd_domain() helper, which probes the live session type via launchctl and returns gui/<uid> on an Aqua GUI runner instead of the user/<uid> the test hardcodes into its plist path fixture. Pin the domain the same way 9c155f3 pinned it for the refresh test: stub _launchctl_session_managername to force headless, and make the launchctl print probe (used by _launchd_domain_for_existing_job) fail so domain resolution falls through to _launchd_domain() instead of matching an already-loaded gui/<uid> job. Unrelated to the daemon-pool py3.14 fix in this branch; found while tracking down PR #53's slice-4/8 CI failure. --------- Co-authored-by: Sahil (AI) <266772320+sahilm-ai@users.noreply.github.com>
…cutor (#53) * fix(tools): support CPython 3.14 WorkerContext in DaemonThreadPoolExecutor CPython 3.14 replaced ThreadPoolExecutor's (initializer, initargs) attribute pair + 4-arg _worker free function with a WorkerContext object built via prepare_context()/_create_worker_context(). Our DaemonThreadPoolExecutor._adjust_thread_count mirrored the pre-3.14 internals directly, so on 3.14 every worker spawn raised AttributeError: 'DaemonThreadPoolExecutor' object has no attribute '_initializer' inside the worker thread, silently breaking every batch of 2+ concurrent tool calls (agent/tool_executor.py routes through this pool). Reproduced empirically on the pre-fix code against the local venv (Python 3.14.6, home = /opt/homebrew/opt/python@3.14): $ python3 -c "from tools.daemon_pool import DaemonThreadPoolExecutor; \ DaemonThreadPoolExecutor(max_workers=2).submit(lambda: 1+1).result()" AttributeError: 'DaemonThreadPoolExecutor' object has no attribute '_initializer' CI runs Python 3.13 (pyproject.toml: requires-python = ">=3.11,<3.14"), so this never surfaced there — only on interpreters that have since moved to 3.14. Fix: branch on sys.version_info at import time and mirror whichever shape of _worker/_adjust_thread_count the running interpreter actually has (confirmed against 3.14.6's concurrent.futures.thread source: the 3.14 _adjust_thread_count calls _worker(weakref, self._create_worker_context(), work_queue), the legacy shape calls _worker(weakref, work_queue, initializer, initargs)). Both branches keep the two behavioral changes that make this pool useful: daemon=True and no _threads_queues registration. Added a regression test (test_many_concurrent_submits_like_tool_executor) reproducing the tool-executor's concurrent-submit shape; verified it fails with the exact AttributeError above against the pre-fix code and passes against the fix, on both Python 3.11 (uv venv, officially supported) and 3.14.6 (local dev venv). * fix(test): pin launchd domain in restart-recovery test to survive Aqua CI runners test_launchd_restart_boots_out_stale_registration_before_bootstrap called the real _launchd_domain() helper, which probes the live session type via launchctl and returns gui/<uid> on an Aqua GUI runner instead of the user/<uid> the test hardcodes into its plist path fixture. Pin the domain the same way 9c155f3 pinned it for the refresh test: stub _launchctl_session_managername to force headless, and make the launchctl print probe (used by _launchd_domain_for_existing_job) fail so domain resolution falls through to _launchd_domain() instead of matching an already-loaded gui/<uid> job. Unrelated to the daemon-pool py3.14 fix in this branch; found while tracking down PR #53's slice-4/8 CI failure. --------- Co-authored-by: Sahil (AI) <266772320+sahilm-ai@users.noreply.github.com>
…cutor (#53) * fix(tools): support CPython 3.14 WorkerContext in DaemonThreadPoolExecutor CPython 3.14 replaced ThreadPoolExecutor's (initializer, initargs) attribute pair + 4-arg _worker free function with a WorkerContext object built via prepare_context()/_create_worker_context(). Our DaemonThreadPoolExecutor._adjust_thread_count mirrored the pre-3.14 internals directly, so on 3.14 every worker spawn raised AttributeError: 'DaemonThreadPoolExecutor' object has no attribute '_initializer' inside the worker thread, silently breaking every batch of 2+ concurrent tool calls (agent/tool_executor.py routes through this pool). Reproduced empirically on the pre-fix code against the local venv (Python 3.14.6, home = /opt/homebrew/opt/python@3.14): $ python3 -c "from tools.daemon_pool import DaemonThreadPoolExecutor; \ DaemonThreadPoolExecutor(max_workers=2).submit(lambda: 1+1).result()" AttributeError: 'DaemonThreadPoolExecutor' object has no attribute '_initializer' CI runs Python 3.13 (pyproject.toml: requires-python = ">=3.11,<3.14"), so this never surfaced there — only on interpreters that have since moved to 3.14. Fix: branch on sys.version_info at import time and mirror whichever shape of _worker/_adjust_thread_count the running interpreter actually has (confirmed against 3.14.6's concurrent.futures.thread source: the 3.14 _adjust_thread_count calls _worker(weakref, self._create_worker_context(), work_queue), the legacy shape calls _worker(weakref, work_queue, initializer, initargs)). Both branches keep the two behavioral changes that make this pool useful: daemon=True and no _threads_queues registration. Added a regression test (test_many_concurrent_submits_like_tool_executor) reproducing the tool-executor's concurrent-submit shape; verified it fails with the exact AttributeError above against the pre-fix code and passes against the fix, on both Python 3.11 (uv venv, officially supported) and 3.14.6 (local dev venv). * fix(test): pin launchd domain in restart-recovery test to survive Aqua CI runners test_launchd_restart_boots_out_stale_registration_before_bootstrap called the real _launchd_domain() helper, which probes the live session type via launchctl and returns gui/<uid> on an Aqua GUI runner instead of the user/<uid> the test hardcodes into its plist path fixture. Pin the domain the same way 9c155f3 pinned it for the refresh test: stub _launchctl_session_managername to force headless, and make the launchctl print probe (used by _launchd_domain_for_existing_job) fail so domain resolution falls through to _launchd_domain() instead of matching an already-loaded gui/<uid> job. Unrelated to the daemon-pool py3.14 fix in this branch; found while tracking down PR #53's slice-4/8 CI failure. --------- Co-authored-by: Sahil (AI) <266772320+sahilm-ai@users.noreply.github.com>
…cutor (#53) * fix(tools): support CPython 3.14 WorkerContext in DaemonThreadPoolExecutor CPython 3.14 replaced ThreadPoolExecutor's (initializer, initargs) attribute pair + 4-arg _worker free function with a WorkerContext object built via prepare_context()/_create_worker_context(). Our DaemonThreadPoolExecutor._adjust_thread_count mirrored the pre-3.14 internals directly, so on 3.14 every worker spawn raised AttributeError: 'DaemonThreadPoolExecutor' object has no attribute '_initializer' inside the worker thread, silently breaking every batch of 2+ concurrent tool calls (agent/tool_executor.py routes through this pool). Reproduced empirically on the pre-fix code against the local venv (Python 3.14.6, home = /opt/homebrew/opt/python@3.14): $ python3 -c "from tools.daemon_pool import DaemonThreadPoolExecutor; \ DaemonThreadPoolExecutor(max_workers=2).submit(lambda: 1+1).result()" AttributeError: 'DaemonThreadPoolExecutor' object has no attribute '_initializer' CI runs Python 3.13 (pyproject.toml: requires-python = ">=3.11,<3.14"), so this never surfaced there — only on interpreters that have since moved to 3.14. Fix: branch on sys.version_info at import time and mirror whichever shape of _worker/_adjust_thread_count the running interpreter actually has (confirmed against 3.14.6's concurrent.futures.thread source: the 3.14 _adjust_thread_count calls _worker(weakref, self._create_worker_context(), work_queue), the legacy shape calls _worker(weakref, work_queue, initializer, initargs)). Both branches keep the two behavioral changes that make this pool useful: daemon=True and no _threads_queues registration. Added a regression test (test_many_concurrent_submits_like_tool_executor) reproducing the tool-executor's concurrent-submit shape; verified it fails with the exact AttributeError above against the pre-fix code and passes against the fix, on both Python 3.11 (uv venv, officially supported) and 3.14.6 (local dev venv). * fix(test): pin launchd domain in restart-recovery test to survive Aqua CI runners test_launchd_restart_boots_out_stale_registration_before_bootstrap called the real _launchd_domain() helper, which probes the live session type via launchctl and returns gui/<uid> on an Aqua GUI runner instead of the user/<uid> the test hardcodes into its plist path fixture. Pin the domain the same way 9c155f3 pinned it for the refresh test: stub _launchctl_session_managername to force headless, and make the launchctl print probe (used by _launchd_domain_for_existing_job) fail so domain resolution falls through to _launchd_domain() instead of matching an already-loaded gui/<uid> job. Unrelated to the daemon-pool py3.14 fix in this branch; found while tracking down PR #53's slice-4/8 CI failure. --------- Co-authored-by: Sahil (AI) <266772320+sahilm-ai@users.noreply.github.com>
…cutor (#53) * fix(tools): support CPython 3.14 WorkerContext in DaemonThreadPoolExecutor CPython 3.14 replaced ThreadPoolExecutor's (initializer, initargs) attribute pair + 4-arg _worker free function with a WorkerContext object built via prepare_context()/_create_worker_context(). Our DaemonThreadPoolExecutor._adjust_thread_count mirrored the pre-3.14 internals directly, so on 3.14 every worker spawn raised AttributeError: 'DaemonThreadPoolExecutor' object has no attribute '_initializer' inside the worker thread, silently breaking every batch of 2+ concurrent tool calls (agent/tool_executor.py routes through this pool). Reproduced empirically on the pre-fix code against the local venv (Python 3.14.6, home = /opt/homebrew/opt/python@3.14): $ python3 -c "from tools.daemon_pool import DaemonThreadPoolExecutor; \ DaemonThreadPoolExecutor(max_workers=2).submit(lambda: 1+1).result()" AttributeError: 'DaemonThreadPoolExecutor' object has no attribute '_initializer' CI runs Python 3.13 (pyproject.toml: requires-python = ">=3.11,<3.14"), so this never surfaced there — only on interpreters that have since moved to 3.14. Fix: branch on sys.version_info at import time and mirror whichever shape of _worker/_adjust_thread_count the running interpreter actually has (confirmed against 3.14.6's concurrent.futures.thread source: the 3.14 _adjust_thread_count calls _worker(weakref, self._create_worker_context(), work_queue), the legacy shape calls _worker(weakref, work_queue, initializer, initargs)). Both branches keep the two behavioral changes that make this pool useful: daemon=True and no _threads_queues registration. Added a regression test (test_many_concurrent_submits_like_tool_executor) reproducing the tool-executor's concurrent-submit shape; verified it fails with the exact AttributeError above against the pre-fix code and passes against the fix, on both Python 3.11 (uv venv, officially supported) and 3.14.6 (local dev venv). * fix(test): pin launchd domain in restart-recovery test to survive Aqua CI runners test_launchd_restart_boots_out_stale_registration_before_bootstrap called the real _launchd_domain() helper, which probes the live session type via launchctl and returns gui/<uid> on an Aqua GUI runner instead of the user/<uid> the test hardcodes into its plist path fixture. Pin the domain the same way 9c155f3 pinned it for the refresh test: stub _launchctl_session_managername to force headless, and make the launchctl print probe (used by _launchd_domain_for_existing_job) fail so domain resolution falls through to _launchd_domain() instead of matching an already-loaded gui/<uid> job. Unrelated to the daemon-pool py3.14 fix in this branch; found while tracking down PR #53's slice-4/8 CI failure. --------- Co-authored-by: Sahil (AI) <266772320+sahilm-ai@users.noreply.github.com>
…cutor (#53) * fix(tools): support CPython 3.14 WorkerContext in DaemonThreadPoolExecutor CPython 3.14 replaced ThreadPoolExecutor's (initializer, initargs) attribute pair + 4-arg _worker free function with a WorkerContext object built via prepare_context()/_create_worker_context(). Our DaemonThreadPoolExecutor._adjust_thread_count mirrored the pre-3.14 internals directly, so on 3.14 every worker spawn raised AttributeError: 'DaemonThreadPoolExecutor' object has no attribute '_initializer' inside the worker thread, silently breaking every batch of 2+ concurrent tool calls (agent/tool_executor.py routes through this pool). Reproduced empirically on the pre-fix code against the local venv (Python 3.14.6, home = /opt/homebrew/opt/python@3.14): $ python3 -c "from tools.daemon_pool import DaemonThreadPoolExecutor; \ DaemonThreadPoolExecutor(max_workers=2).submit(lambda: 1+1).result()" AttributeError: 'DaemonThreadPoolExecutor' object has no attribute '_initializer' CI runs Python 3.13 (pyproject.toml: requires-python = ">=3.11,<3.14"), so this never surfaced there — only on interpreters that have since moved to 3.14. Fix: branch on sys.version_info at import time and mirror whichever shape of _worker/_adjust_thread_count the running interpreter actually has (confirmed against 3.14.6's concurrent.futures.thread source: the 3.14 _adjust_thread_count calls _worker(weakref, self._create_worker_context(), work_queue), the legacy shape calls _worker(weakref, work_queue, initializer, initargs)). Both branches keep the two behavioral changes that make this pool useful: daemon=True and no _threads_queues registration. Added a regression test (test_many_concurrent_submits_like_tool_executor) reproducing the tool-executor's concurrent-submit shape; verified it fails with the exact AttributeError above against the pre-fix code and passes against the fix, on both Python 3.11 (uv venv, officially supported) and 3.14.6 (local dev venv). * fix(test): pin launchd domain in restart-recovery test to survive Aqua CI runners test_launchd_restart_boots_out_stale_registration_before_bootstrap called the real _launchd_domain() helper, which probes the live session type via launchctl and returns gui/<uid> on an Aqua GUI runner instead of the user/<uid> the test hardcodes into its plist path fixture. Pin the domain the same way 9c155f3 pinned it for the refresh test: stub _launchctl_session_managername to force headless, and make the launchctl print probe (used by _launchd_domain_for_existing_job) fail so domain resolution falls through to _launchd_domain() instead of matching an already-loaded gui/<uid> job. Unrelated to the daemon-pool py3.14 fix in this branch; found while tracking down PR #53's slice-4/8 CI failure. --------- Co-authored-by: Sahil (AI) <266772320+sahilm-ai@users.noreply.github.com>
…cutor (#53) * fix(tools): support CPython 3.14 WorkerContext in DaemonThreadPoolExecutor CPython 3.14 replaced ThreadPoolExecutor's (initializer, initargs) attribute pair + 4-arg _worker free function with a WorkerContext object built via prepare_context()/_create_worker_context(). Our DaemonThreadPoolExecutor._adjust_thread_count mirrored the pre-3.14 internals directly, so on 3.14 every worker spawn raised AttributeError: 'DaemonThreadPoolExecutor' object has no attribute '_initializer' inside the worker thread, silently breaking every batch of 2+ concurrent tool calls (agent/tool_executor.py routes through this pool). Reproduced empirically on the pre-fix code against the local venv (Python 3.14.6, home = /opt/homebrew/opt/python@3.14): $ python3 -c "from tools.daemon_pool import DaemonThreadPoolExecutor; \ DaemonThreadPoolExecutor(max_workers=2).submit(lambda: 1+1).result()" AttributeError: 'DaemonThreadPoolExecutor' object has no attribute '_initializer' CI runs Python 3.13 (pyproject.toml: requires-python = ">=3.11,<3.14"), so this never surfaced there — only on interpreters that have since moved to 3.14. Fix: branch on sys.version_info at import time and mirror whichever shape of _worker/_adjust_thread_count the running interpreter actually has (confirmed against 3.14.6's concurrent.futures.thread source: the 3.14 _adjust_thread_count calls _worker(weakref, self._create_worker_context(), work_queue), the legacy shape calls _worker(weakref, work_queue, initializer, initargs)). Both branches keep the two behavioral changes that make this pool useful: daemon=True and no _threads_queues registration. Added a regression test (test_many_concurrent_submits_like_tool_executor) reproducing the tool-executor's concurrent-submit shape; verified it fails with the exact AttributeError above against the pre-fix code and passes against the fix, on both Python 3.11 (uv venv, officially supported) and 3.14.6 (local dev venv). * fix(test): pin launchd domain in restart-recovery test to survive Aqua CI runners test_launchd_restart_boots_out_stale_registration_before_bootstrap called the real _launchd_domain() helper, which probes the live session type via launchctl and returns gui/<uid> on an Aqua GUI runner instead of the user/<uid> the test hardcodes into its plist path fixture. Pin the domain the same way 9c155f3 pinned it for the refresh test: stub _launchctl_session_managername to force headless, and make the launchctl print probe (used by _launchd_domain_for_existing_job) fail so domain resolution falls through to _launchd_domain() instead of matching an already-loaded gui/<uid> job. Unrelated to the daemon-pool py3.14 fix in this branch; found while tracking down PR #53's slice-4/8 CI failure. --------- Co-authored-by: Sahil (AI) <266772320+sahilm-ai@users.noreply.github.com>
…cutor (#53) * fix(tools): support CPython 3.14 WorkerContext in DaemonThreadPoolExecutor CPython 3.14 replaced ThreadPoolExecutor's (initializer, initargs) attribute pair + 4-arg _worker free function with a WorkerContext object built via prepare_context()/_create_worker_context(). Our DaemonThreadPoolExecutor._adjust_thread_count mirrored the pre-3.14 internals directly, so on 3.14 every worker spawn raised AttributeError: 'DaemonThreadPoolExecutor' object has no attribute '_initializer' inside the worker thread, silently breaking every batch of 2+ concurrent tool calls (agent/tool_executor.py routes through this pool). Reproduced empirically on the pre-fix code against the local venv (Python 3.14.6, home = /opt/homebrew/opt/python@3.14): $ python3 -c "from tools.daemon_pool import DaemonThreadPoolExecutor; \ DaemonThreadPoolExecutor(max_workers=2).submit(lambda: 1+1).result()" AttributeError: 'DaemonThreadPoolExecutor' object has no attribute '_initializer' CI runs Python 3.13 (pyproject.toml: requires-python = ">=3.11,<3.14"), so this never surfaced there — only on interpreters that have since moved to 3.14. Fix: branch on sys.version_info at import time and mirror whichever shape of _worker/_adjust_thread_count the running interpreter actually has (confirmed against 3.14.6's concurrent.futures.thread source: the 3.14 _adjust_thread_count calls _worker(weakref, self._create_worker_context(), work_queue), the legacy shape calls _worker(weakref, work_queue, initializer, initargs)). Both branches keep the two behavioral changes that make this pool useful: daemon=True and no _threads_queues registration. Added a regression test (test_many_concurrent_submits_like_tool_executor) reproducing the tool-executor's concurrent-submit shape; verified it fails with the exact AttributeError above against the pre-fix code and passes against the fix, on both Python 3.11 (uv venv, officially supported) and 3.14.6 (local dev venv). * fix(test): pin launchd domain in restart-recovery test to survive Aqua CI runners test_launchd_restart_boots_out_stale_registration_before_bootstrap called the real _launchd_domain() helper, which probes the live session type via launchctl and returns gui/<uid> on an Aqua GUI runner instead of the user/<uid> the test hardcodes into its plist path fixture. Pin the domain the same way 9c155f3 pinned it for the refresh test: stub _launchctl_session_managername to force headless, and make the launchctl print probe (used by _launchd_domain_for_existing_job) fail so domain resolution falls through to _launchd_domain() instead of matching an already-loaded gui/<uid> job. Unrelated to the daemon-pool py3.14 fix in this branch; found while tracking down PR #53's slice-4/8 CI failure. --------- Co-authored-by: Sahil (AI) <266772320+sahilm-ai@users.noreply.github.com>
…cutor (#53) * fix(tools): support CPython 3.14 WorkerContext in DaemonThreadPoolExecutor CPython 3.14 replaced ThreadPoolExecutor's (initializer, initargs) attribute pair + 4-arg _worker free function with a WorkerContext object built via prepare_context()/_create_worker_context(). Our DaemonThreadPoolExecutor._adjust_thread_count mirrored the pre-3.14 internals directly, so on 3.14 every worker spawn raised AttributeError: 'DaemonThreadPoolExecutor' object has no attribute '_initializer' inside the worker thread, silently breaking every batch of 2+ concurrent tool calls (agent/tool_executor.py routes through this pool). Reproduced empirically on the pre-fix code against the local venv (Python 3.14.6, home = /opt/homebrew/opt/python@3.14): $ python3 -c "from tools.daemon_pool import DaemonThreadPoolExecutor; \ DaemonThreadPoolExecutor(max_workers=2).submit(lambda: 1+1).result()" AttributeError: 'DaemonThreadPoolExecutor' object has no attribute '_initializer' CI runs Python 3.13 (pyproject.toml: requires-python = ">=3.11,<3.14"), so this never surfaced there — only on interpreters that have since moved to 3.14. Fix: branch on sys.version_info at import time and mirror whichever shape of _worker/_adjust_thread_count the running interpreter actually has (confirmed against 3.14.6's concurrent.futures.thread source: the 3.14 _adjust_thread_count calls _worker(weakref, self._create_worker_context(), work_queue), the legacy shape calls _worker(weakref, work_queue, initializer, initargs)). Both branches keep the two behavioral changes that make this pool useful: daemon=True and no _threads_queues registration. Added a regression test (test_many_concurrent_submits_like_tool_executor) reproducing the tool-executor's concurrent-submit shape; verified it fails with the exact AttributeError above against the pre-fix code and passes against the fix, on both Python 3.11 (uv venv, officially supported) and 3.14.6 (local dev venv). * fix(test): pin launchd domain in restart-recovery test to survive Aqua CI runners test_launchd_restart_boots_out_stale_registration_before_bootstrap called the real _launchd_domain() helper, which probes the live session type via launchctl and returns gui/<uid> on an Aqua GUI runner instead of the user/<uid> the test hardcodes into its plist path fixture. Pin the domain the same way 9c155f3 pinned it for the refresh test: stub _launchctl_session_managername to force headless, and make the launchctl print probe (used by _launchd_domain_for_existing_job) fail so domain resolution falls through to _launchd_domain() instead of matching an already-loaded gui/<uid> job. Unrelated to the daemon-pool py3.14 fix in this branch; found while tracking down PR #53's slice-4/8 CI failure. --------- Co-authored-by: Sahil (AI) <266772320+sahilm-ai@users.noreply.github.com>
…cutor (#53) * fix(tools): support CPython 3.14 WorkerContext in DaemonThreadPoolExecutor CPython 3.14 replaced ThreadPoolExecutor's (initializer, initargs) attribute pair + 4-arg _worker free function with a WorkerContext object built via prepare_context()/_create_worker_context(). Our DaemonThreadPoolExecutor._adjust_thread_count mirrored the pre-3.14 internals directly, so on 3.14 every worker spawn raised AttributeError: 'DaemonThreadPoolExecutor' object has no attribute '_initializer' inside the worker thread, silently breaking every batch of 2+ concurrent tool calls (agent/tool_executor.py routes through this pool). Reproduced empirically on the pre-fix code against the local venv (Python 3.14.6, home = /opt/homebrew/opt/python@3.14): $ python3 -c "from tools.daemon_pool import DaemonThreadPoolExecutor; \ DaemonThreadPoolExecutor(max_workers=2).submit(lambda: 1+1).result()" AttributeError: 'DaemonThreadPoolExecutor' object has no attribute '_initializer' CI runs Python 3.13 (pyproject.toml: requires-python = ">=3.11,<3.14"), so this never surfaced there — only on interpreters that have since moved to 3.14. Fix: branch on sys.version_info at import time and mirror whichever shape of _worker/_adjust_thread_count the running interpreter actually has (confirmed against 3.14.6's concurrent.futures.thread source: the 3.14 _adjust_thread_count calls _worker(weakref, self._create_worker_context(), work_queue), the legacy shape calls _worker(weakref, work_queue, initializer, initargs)). Both branches keep the two behavioral changes that make this pool useful: daemon=True and no _threads_queues registration. Added a regression test (test_many_concurrent_submits_like_tool_executor) reproducing the tool-executor's concurrent-submit shape; verified it fails with the exact AttributeError above against the pre-fix code and passes against the fix, on both Python 3.11 (uv venv, officially supported) and 3.14.6 (local dev venv). * fix(test): pin launchd domain in restart-recovery test to survive Aqua CI runners test_launchd_restart_boots_out_stale_registration_before_bootstrap called the real _launchd_domain() helper, which probes the live session type via launchctl and returns gui/<uid> on an Aqua GUI runner instead of the user/<uid> the test hardcodes into its plist path fixture. Pin the domain the same way 9c155f3 pinned it for the refresh test: stub _launchctl_session_managername to force headless, and make the launchctl print probe (used by _launchd_domain_for_existing_job) fail so domain resolution falls through to _launchd_domain() instead of matching an already-loaded gui/<uid> job. Unrelated to the daemon-pool py3.14 fix in this branch; found while tracking down PR #53's slice-4/8 CI failure. --------- Co-authored-by: Sahil (AI) <266772320+sahilm-ai@users.noreply.github.com>
Problem
CPython 3.14 replaced
ThreadPoolExecutor's(initializer, initargs)attribute pair + 4-arg_workerfree function with aWorkerContextobject built viaprepare_context()/self._create_worker_context().DaemonThreadPoolExecutor._adjust_thread_countmirrored the pre-3.14 internals directly (readingself._initializer/self._initargs), so on 3.14 every worker spawn raised:inside the worker thread — silently breaking every batch of 2+ concurrent tool calls, since
agent/tool_executor.pyroutes concurrent tool execution through this pool.CI runs Python 3.13 (
requires-python = ">=3.11,<3.14"), so this never surfaced there. It only shows up on interpreters that have moved to 3.14 (e.g. local dev venvs pinned viapyvenv.cfgto/opt/homebrew/opt/python@3.14).Empirical repro (pre-fix, on 3.14.6)
Fix
Branch on
sys.version_infoat import time and mirror whichever shape of_worker/_adjust_thread_countthe running interpreter actually has, confirmed against 3.14.6'sconcurrent.futures.threadsource directly:_worker(weakref, self._create_worker_context(), work_queue)_worker(weakref, work_queue, initializer, initargs)Both branches preserve the two behavioral changes that make this pool useful:
daemon=Trueand no_threads_queuesregistration (so a wedged worker never blocks interpreter exit).Testing
Added
test_many_concurrent_submits_like_tool_executor, reproducing the tool-executor's concurrent-submit shape (more submissions thanmax_workers, forcing multiple_adjust_thread_countcalls).AttributeErrorabove against the pre-fix code on 3.14.6.requires-python)ruff check/ruff format --checkclean on both changed files.ty checkshows only pre-existing baseline diagnostics unrelated to this change (present, and worse, on the unmodified file).Summary by CodeRabbit
Bug Fixes
Tests