Skip to content

fix(tools): support CPython 3.14 WorkerContext in DaemonThreadPoolExecutor - #53

Merged
sahilm-ai merged 2 commits into
mainfrom
fix/daemon-pool-py314
Jul 3, 2026
Merged

sahilm-ai merged 2 commits into
mainfrom
fix/daemon-pool-py314

Conversation

@sahilm-ai

@sahilm-ai sahilm-ai commented Jul 3, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

CPython 3.14 replaced ThreadPoolExecutor's (initializer, initargs) attribute pair + 4-arg _worker free function with a WorkerContext object built via prepare_context() / self._create_worker_context(). DaemonThreadPoolExecutor._adjust_thread_count mirrored the pre-3.14 internals directly (reading self._initializer / self._initargs), so on 3.14 every worker spawn raised:

AttributeError: 'DaemonThreadPoolExecutor' object has no attribute '_initializer'

inside the worker thread — silently breaking every batch of 2+ concurrent tool calls, since agent/tool_executor.py routes concurrent tool execution through this pool.

CI runs Python 3.13 (requires-python = ">=3.11,<3.14"), so this never surfaced there. It only shows up on interpreters that have moved to 3.14 (e.g. local dev venvs pinned via pyvenv.cfg to /opt/homebrew/opt/python@3.14).

Empirical repro (pre-fix, on 3.14.6)

$ python3 -c "from tools.daemon_pool import DaemonThreadPoolExecutor; \
    DaemonThreadPoolExecutor(max_workers=2).submit(lambda: 1+1).result()"
AttributeError: 'DaemonThreadPoolExecutor' object has no attribute '_initializer'

Fix

Branch on sys.version_info at import time and mirror whichever shape of _worker/_adjust_thread_count the running interpreter actually has, confirmed against 3.14.6's concurrent.futures.thread source directly:

  • >=3.14: _worker(weakref, self._create_worker_context(), work_queue)
  • <3.14 (legacy): _worker(weakref, work_queue, initializer, initargs)

Both branches preserve the two behavioral changes that make this pool useful: daemon=True and no _threads_queues registration (so a wedged worker never blocks interpreter exit).

Testing

Added test_many_concurrent_submits_like_tool_executor, reproducing the tool-executor's concurrent-submit shape (more submissions than max_workers, forcing multiple _adjust_thread_count calls).

  • Verified it fails with the exact AttributeError above against the pre-fix code on 3.14.6.
  • Verified all 5 tests pass against the fix on both:
    • Python 3.11.15 (uv-managed venv — the officially supported interpreter per requires-python)
    • Python 3.14.6 (local dev venv)
  • ruff check / ruff format --check clean on both changed files.
  • ty check shows only pre-existing baseline diagnostics unrelated to this change (present, and worse, on the unmodified file).

Summary by CodeRabbit

  • Bug Fixes

    • Improved reliability of background task execution under heavy concurrent submission loads.
    • Fixed worker startup/thread initialization behavior to align with Python runtime expectations across supported versions.
  • Tests

    • Added a regression test covering high-concurrency task submissions to prevent future executor startup failures.
    • Updated a gateway service test to make LaunchAgent domain resolution deterministic in headless runs.

…cutor

CPython 3.14 replaced ThreadPoolExecutor's (initializer, initargs)
attribute pair + 4-arg _worker free function with a WorkerContext
object built via prepare_context()/_create_worker_context(). Our
DaemonThreadPoolExecutor._adjust_thread_count mirrored the pre-3.14
internals directly, so on 3.14 every worker spawn raised

  AttributeError: 'DaemonThreadPoolExecutor' object has no attribute
  '_initializer'

inside the worker thread, silently breaking every batch of 2+
concurrent tool calls (agent/tool_executor.py routes through this
pool). Reproduced empirically on the pre-fix code against the local
venv (Python 3.14.6, home = /opt/homebrew/opt/python@3.14):

  $ python3 -c "from tools.daemon_pool import DaemonThreadPoolExecutor; \
    DaemonThreadPoolExecutor(max_workers=2).submit(lambda: 1+1).result()"
  AttributeError: 'DaemonThreadPoolExecutor' object has no attribute '_initializer'

CI runs Python 3.13 (pyproject.toml: requires-python = ">=3.11,<3.14"),
so this never surfaced there — only on interpreters that have since
moved to 3.14.

Fix: branch on sys.version_info at import time and mirror whichever
shape of _worker/_adjust_thread_count the running interpreter actually
has (confirmed against 3.14.6's concurrent.futures.thread source: the
3.14 _adjust_thread_count calls _worker(weakref, self._create_worker_context(),
work_queue), the legacy shape calls _worker(weakref, work_queue,
initializer, initargs)). Both branches keep the two behavioral changes
that make this pool useful: daemon=True and no _threads_queues
registration.

Added a regression test (test_many_concurrent_submits_like_tool_executor)
reproducing the tool-executor's concurrent-submit shape; verified it
fails with the exact AttributeError above against the pre-fix code and
passes against the fix, on both Python 3.11 (uv venv, officially
supported) and 3.14.6 (local dev venv).
@coderabbitai

coderabbitai Bot commented Jul 3, 2026 •

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 2a2d3722-1f48-4bae-bc51-4b60a7060fe2

📥 Commits

Reviewing files that changed from the base of the PR and between adc5fd1 and d720e62.

📒 Files selected for processing (1)
  • tests/hermes_cli/test_gateway_service.py

📝 Walkthrough

Walkthrough

This PR updates daemon thread pool worker spawning to match CPython’s private _worker API across Python versions and adds a regression test for concurrent submissions. It also makes one gateway service restart test use deterministic launchd domain resolution and subprocess call assertions.

Changes

Daemon Pool Worker Compatibility

Layer / File(s) Summary
Version-aware worker spawning
tools/daemon_pool.py
Adds _PY314_WORKER_CONTEXT and _spawn_daemon_worker, conditionally importing and invoking the matching private _worker signature, and routes _adjust_thread_count through the helper.
Concurrent submit regression test
tests/tools/test_daemon_pool.py
Adds test_many_concurrent_submits_like_tool_executor, submitting 12 tasks to a 4-worker pool and asserting all futures complete with expected squared results.

Launchd Restart Test Determinism

Layer / File(s) Summary
Pinned launchd domain and filtered probes
tests/hermes_cli/test_gateway_service.py
Forces headless launchd domain computation to user/<uid> and excludes launchctl print probes from recorded subprocess calls so the restart assertions only cover the action sequence.

Estimated code review effort: 4 (Complex) | ~35 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Executor as DaemonThreadPoolExecutor
  participant Adjust as _adjust_thread_count
  participant Spawn as _spawn_daemon_worker
  participant Worker as _worker

  Executor->>Adjust: request a new daemon thread
  Adjust->>Spawn: _spawn_daemon_worker(self, thread_name, weakref_cb)
  Spawn->>Worker: start thread with version-matched arguments
Loading

Possibly related PRs

  • sahilm-ti/hermes-agent#48: Also updates test_gateway_service.py to pin launchd domain behavior and adjust launchctl subprocess expectations.
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main fix: adding CPython 3.14 WorkerContext support to DaemonThreadPoolExecutor.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/daemon-pool-py314

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

…a CI runners

test_launchd_restart_boots_out_stale_registration_before_bootstrap called
the real _launchd_domain() helper, which probes the live session type via
launchctl and returns gui/<uid> on an Aqua GUI runner instead of the
user/<uid> the test hardcodes into its plist path fixture. Pin the domain
the same way 9c155f3 pinned it for the refresh test: stub
_launchctl_session_managername to force headless, and make the launchctl
print probe (used by _launchd_domain_for_existing_job) fail so domain
resolution falls through to _launchd_domain() instead of matching an
already-loaded gui/<uid> job.

Unrelated to the daemon-pool py3.14 fix in this branch; found while
tracking down PR #53's slice-4/8 CI failure.
@sahilm-ai
sahilm-ai merged commit 157b576 into main Jul 3, 2026
31 checks passed
@sahilm-ai
sahilm-ai deleted the fix/daemon-pool-py314 branch July 3, 2026 11:59
sahilm-ti pushed a commit that referenced this pull request Jul 9, 2026
…cutor (#53)

* fix(tools): support CPython 3.14 WorkerContext in DaemonThreadPoolExecutor

CPython 3.14 replaced ThreadPoolExecutor's (initializer, initargs)
attribute pair + 4-arg _worker free function with a WorkerContext
object built via prepare_context()/_create_worker_context(). Our
DaemonThreadPoolExecutor._adjust_thread_count mirrored the pre-3.14
internals directly, so on 3.14 every worker spawn raised

  AttributeError: 'DaemonThreadPoolExecutor' object has no attribute
  '_initializer'

inside the worker thread, silently breaking every batch of 2+
concurrent tool calls (agent/tool_executor.py routes through this
pool). Reproduced empirically on the pre-fix code against the local
venv (Python 3.14.6, home = /opt/homebrew/opt/python@3.14):

  $ python3 -c "from tools.daemon_pool import DaemonThreadPoolExecutor; \
    DaemonThreadPoolExecutor(max_workers=2).submit(lambda: 1+1).result()"
  AttributeError: 'DaemonThreadPoolExecutor' object has no attribute '_initializer'

CI runs Python 3.13 (pyproject.toml: requires-python = ">=3.11,<3.14"),
so this never surfaced there — only on interpreters that have since
moved to 3.14.

Fix: branch on sys.version_info at import time and mirror whichever
shape of _worker/_adjust_thread_count the running interpreter actually
has (confirmed against 3.14.6's concurrent.futures.thread source: the
3.14 _adjust_thread_count calls _worker(weakref, self._create_worker_context(),
work_queue), the legacy shape calls _worker(weakref, work_queue,
initializer, initargs)). Both branches keep the two behavioral changes
that make this pool useful: daemon=True and no _threads_queues
registration.

Added a regression test (test_many_concurrent_submits_like_tool_executor)
reproducing the tool-executor's concurrent-submit shape; verified it
fails with the exact AttributeError above against the pre-fix code and
passes against the fix, on both Python 3.11 (uv venv, officially
supported) and 3.14.6 (local dev venv).

* fix(test): pin launchd domain in restart-recovery test to survive Aqua CI runners

test_launchd_restart_boots_out_stale_registration_before_bootstrap called
the real _launchd_domain() helper, which probes the live session type via
launchctl and returns gui/<uid> on an Aqua GUI runner instead of the
user/<uid> the test hardcodes into its plist path fixture. Pin the domain
the same way 9c155f3 pinned it for the refresh test: stub
_launchctl_session_managername to force headless, and make the launchctl
print probe (used by _launchd_domain_for_existing_job) fail so domain
resolution falls through to _launchd_domain() instead of matching an
already-loaded gui/<uid> job.

Unrelated to the daemon-pool py3.14 fix in this branch; found while
tracking down PR #53's slice-4/8 CI failure.

---------

Co-authored-by: Sahil (AI) <266772320+sahilm-ai@users.noreply.github.com>
sahilm-ti pushed a commit that referenced this pull request Jul 10, 2026
…cutor (#53)

* fix(tools): support CPython 3.14 WorkerContext in DaemonThreadPoolExecutor

CPython 3.14 replaced ThreadPoolExecutor's (initializer, initargs)
attribute pair + 4-arg _worker free function with a WorkerContext
object built via prepare_context()/_create_worker_context(). Our
DaemonThreadPoolExecutor._adjust_thread_count mirrored the pre-3.14
internals directly, so on 3.14 every worker spawn raised

  AttributeError: 'DaemonThreadPoolExecutor' object has no attribute
  '_initializer'

inside the worker thread, silently breaking every batch of 2+
concurrent tool calls (agent/tool_executor.py routes through this
pool). Reproduced empirically on the pre-fix code against the local
venv (Python 3.14.6, home = /opt/homebrew/opt/python@3.14):

  $ python3 -c "from tools.daemon_pool import DaemonThreadPoolExecutor; \
    DaemonThreadPoolExecutor(max_workers=2).submit(lambda: 1+1).result()"
  AttributeError: 'DaemonThreadPoolExecutor' object has no attribute '_initializer'

CI runs Python 3.13 (pyproject.toml: requires-python = ">=3.11,<3.14"),
so this never surfaced there — only on interpreters that have since
moved to 3.14.

Fix: branch on sys.version_info at import time and mirror whichever
shape of _worker/_adjust_thread_count the running interpreter actually
has (confirmed against 3.14.6's concurrent.futures.thread source: the
3.14 _adjust_thread_count calls _worker(weakref, self._create_worker_context(),
work_queue), the legacy shape calls _worker(weakref, work_queue,
initializer, initargs)). Both branches keep the two behavioral changes
that make this pool useful: daemon=True and no _threads_queues
registration.

Added a regression test (test_many_concurrent_submits_like_tool_executor)
reproducing the tool-executor's concurrent-submit shape; verified it
fails with the exact AttributeError above against the pre-fix code and
passes against the fix, on both Python 3.11 (uv venv, officially
supported) and 3.14.6 (local dev venv).

* fix(test): pin launchd domain in restart-recovery test to survive Aqua CI runners

test_launchd_restart_boots_out_stale_registration_before_bootstrap called
the real _launchd_domain() helper, which probes the live session type via
launchctl and returns gui/<uid> on an Aqua GUI runner instead of the
user/<uid> the test hardcodes into its plist path fixture. Pin the domain
the same way 9c155f3 pinned it for the refresh test: stub
_launchctl_session_managername to force headless, and make the launchctl
print probe (used by _launchd_domain_for_existing_job) fail so domain
resolution falls through to _launchd_domain() instead of matching an
already-loaded gui/<uid> job.

Unrelated to the daemon-pool py3.14 fix in this branch; found while
tracking down PR #53's slice-4/8 CI failure.

---------

Co-authored-by: Sahil (AI) <266772320+sahilm-ai@users.noreply.github.com>
sahilm-ti pushed a commit that referenced this pull request Jul 11, 2026
…cutor (#53)

* fix(tools): support CPython 3.14 WorkerContext in DaemonThreadPoolExecutor

CPython 3.14 replaced ThreadPoolExecutor's (initializer, initargs)
attribute pair + 4-arg _worker free function with a WorkerContext
object built via prepare_context()/_create_worker_context(). Our
DaemonThreadPoolExecutor._adjust_thread_count mirrored the pre-3.14
internals directly, so on 3.14 every worker spawn raised

  AttributeError: 'DaemonThreadPoolExecutor' object has no attribute
  '_initializer'

inside the worker thread, silently breaking every batch of 2+
concurrent tool calls (agent/tool_executor.py routes through this
pool). Reproduced empirically on the pre-fix code against the local
venv (Python 3.14.6, home = /opt/homebrew/opt/python@3.14):

  $ python3 -c "from tools.daemon_pool import DaemonThreadPoolExecutor; \
    DaemonThreadPoolExecutor(max_workers=2).submit(lambda: 1+1).result()"
  AttributeError: 'DaemonThreadPoolExecutor' object has no attribute '_initializer'

CI runs Python 3.13 (pyproject.toml: requires-python = ">=3.11,<3.14"),
so this never surfaced there — only on interpreters that have since
moved to 3.14.

Fix: branch on sys.version_info at import time and mirror whichever
shape of _worker/_adjust_thread_count the running interpreter actually
has (confirmed against 3.14.6's concurrent.futures.thread source: the
3.14 _adjust_thread_count calls _worker(weakref, self._create_worker_context(),
work_queue), the legacy shape calls _worker(weakref, work_queue,
initializer, initargs)). Both branches keep the two behavioral changes
that make this pool useful: daemon=True and no _threads_queues
registration.

Added a regression test (test_many_concurrent_submits_like_tool_executor)
reproducing the tool-executor's concurrent-submit shape; verified it
fails with the exact AttributeError above against the pre-fix code and
passes against the fix, on both Python 3.11 (uv venv, officially
supported) and 3.14.6 (local dev venv).

* fix(test): pin launchd domain in restart-recovery test to survive Aqua CI runners

test_launchd_restart_boots_out_stale_registration_before_bootstrap called
the real _launchd_domain() helper, which probes the live session type via
launchctl and returns gui/<uid> on an Aqua GUI runner instead of the
user/<uid> the test hardcodes into its plist path fixture. Pin the domain
the same way 9c155f3 pinned it for the refresh test: stub
_launchctl_session_managername to force headless, and make the launchctl
print probe (used by _launchd_domain_for_existing_job) fail so domain
resolution falls through to _launchd_domain() instead of matching an
already-loaded gui/<uid> job.

Unrelated to the daemon-pool py3.14 fix in this branch; found while
tracking down PR #53's slice-4/8 CI failure.

---------

Co-authored-by: Sahil (AI) <266772320+sahilm-ai@users.noreply.github.com>
sahilm-ti pushed a commit that referenced this pull request Jul 13, 2026
…cutor (#53)

* fix(tools): support CPython 3.14 WorkerContext in DaemonThreadPoolExecutor

CPython 3.14 replaced ThreadPoolExecutor's (initializer, initargs)
attribute pair + 4-arg _worker free function with a WorkerContext
object built via prepare_context()/_create_worker_context(). Our
DaemonThreadPoolExecutor._adjust_thread_count mirrored the pre-3.14
internals directly, so on 3.14 every worker spawn raised

  AttributeError: 'DaemonThreadPoolExecutor' object has no attribute
  '_initializer'

inside the worker thread, silently breaking every batch of 2+
concurrent tool calls (agent/tool_executor.py routes through this
pool). Reproduced empirically on the pre-fix code against the local
venv (Python 3.14.6, home = /opt/homebrew/opt/python@3.14):

  $ python3 -c "from tools.daemon_pool import DaemonThreadPoolExecutor; \
    DaemonThreadPoolExecutor(max_workers=2).submit(lambda: 1+1).result()"
  AttributeError: 'DaemonThreadPoolExecutor' object has no attribute '_initializer'

CI runs Python 3.13 (pyproject.toml: requires-python = ">=3.11,<3.14"),
so this never surfaced there — only on interpreters that have since
moved to 3.14.

Fix: branch on sys.version_info at import time and mirror whichever
shape of _worker/_adjust_thread_count the running interpreter actually
has (confirmed against 3.14.6's concurrent.futures.thread source: the
3.14 _adjust_thread_count calls _worker(weakref, self._create_worker_context(),
work_queue), the legacy shape calls _worker(weakref, work_queue,
initializer, initargs)). Both branches keep the two behavioral changes
that make this pool useful: daemon=True and no _threads_queues
registration.

Added a regression test (test_many_concurrent_submits_like_tool_executor)
reproducing the tool-executor's concurrent-submit shape; verified it
fails with the exact AttributeError above against the pre-fix code and
passes against the fix, on both Python 3.11 (uv venv, officially
supported) and 3.14.6 (local dev venv).

* fix(test): pin launchd domain in restart-recovery test to survive Aqua CI runners

test_launchd_restart_boots_out_stale_registration_before_bootstrap called
the real _launchd_domain() helper, which probes the live session type via
launchctl and returns gui/<uid> on an Aqua GUI runner instead of the
user/<uid> the test hardcodes into its plist path fixture. Pin the domain
the same way 9c155f3 pinned it for the refresh test: stub
_launchctl_session_managername to force headless, and make the launchctl
print probe (used by _launchd_domain_for_existing_job) fail so domain
resolution falls through to _launchd_domain() instead of matching an
already-loaded gui/<uid> job.

Unrelated to the daemon-pool py3.14 fix in this branch; found while
tracking down PR #53's slice-4/8 CI failure.

---------

Co-authored-by: Sahil (AI) <266772320+sahilm-ai@users.noreply.github.com>
sahilm-ti pushed a commit that referenced this pull request Jul 15, 2026
…cutor (#53)

* fix(tools): support CPython 3.14 WorkerContext in DaemonThreadPoolExecutor

CPython 3.14 replaced ThreadPoolExecutor's (initializer, initargs)
attribute pair + 4-arg _worker free function with a WorkerContext
object built via prepare_context()/_create_worker_context(). Our
DaemonThreadPoolExecutor._adjust_thread_count mirrored the pre-3.14
internals directly, so on 3.14 every worker spawn raised

  AttributeError: 'DaemonThreadPoolExecutor' object has no attribute
  '_initializer'

inside the worker thread, silently breaking every batch of 2+
concurrent tool calls (agent/tool_executor.py routes through this
pool). Reproduced empirically on the pre-fix code against the local
venv (Python 3.14.6, home = /opt/homebrew/opt/python@3.14):

  $ python3 -c "from tools.daemon_pool import DaemonThreadPoolExecutor; \
    DaemonThreadPoolExecutor(max_workers=2).submit(lambda: 1+1).result()"
  AttributeError: 'DaemonThreadPoolExecutor' object has no attribute '_initializer'

CI runs Python 3.13 (pyproject.toml: requires-python = ">=3.11,<3.14"),
so this never surfaced there — only on interpreters that have since
moved to 3.14.

Fix: branch on sys.version_info at import time and mirror whichever
shape of _worker/_adjust_thread_count the running interpreter actually
has (confirmed against 3.14.6's concurrent.futures.thread source: the
3.14 _adjust_thread_count calls _worker(weakref, self._create_worker_context(),
work_queue), the legacy shape calls _worker(weakref, work_queue,
initializer, initargs)). Both branches keep the two behavioral changes
that make this pool useful: daemon=True and no _threads_queues
registration.

Added a regression test (test_many_concurrent_submits_like_tool_executor)
reproducing the tool-executor's concurrent-submit shape; verified it
fails with the exact AttributeError above against the pre-fix code and
passes against the fix, on both Python 3.11 (uv venv, officially
supported) and 3.14.6 (local dev venv).

* fix(test): pin launchd domain in restart-recovery test to survive Aqua CI runners

test_launchd_restart_boots_out_stale_registration_before_bootstrap called
the real _launchd_domain() helper, which probes the live session type via
launchctl and returns gui/<uid> on an Aqua GUI runner instead of the
user/<uid> the test hardcodes into its plist path fixture. Pin the domain
the same way 9c155f3 pinned it for the refresh test: stub
_launchctl_session_managername to force headless, and make the launchctl
print probe (used by _launchd_domain_for_existing_job) fail so domain
resolution falls through to _launchd_domain() instead of matching an
already-loaded gui/<uid> job.

Unrelated to the daemon-pool py3.14 fix in this branch; found while
tracking down PR #53's slice-4/8 CI failure.

---------

Co-authored-by: Sahil (AI) <266772320+sahilm-ai@users.noreply.github.com>
sahilm-ti pushed a commit that referenced this pull request Jul 17, 2026
…cutor (#53)

* fix(tools): support CPython 3.14 WorkerContext in DaemonThreadPoolExecutor

CPython 3.14 replaced ThreadPoolExecutor's (initializer, initargs)
attribute pair + 4-arg _worker free function with a WorkerContext
object built via prepare_context()/_create_worker_context(). Our
DaemonThreadPoolExecutor._adjust_thread_count mirrored the pre-3.14
internals directly, so on 3.14 every worker spawn raised

  AttributeError: 'DaemonThreadPoolExecutor' object has no attribute
  '_initializer'

inside the worker thread, silently breaking every batch of 2+
concurrent tool calls (agent/tool_executor.py routes through this
pool). Reproduced empirically on the pre-fix code against the local
venv (Python 3.14.6, home = /opt/homebrew/opt/python@3.14):

  $ python3 -c "from tools.daemon_pool import DaemonThreadPoolExecutor; \
    DaemonThreadPoolExecutor(max_workers=2).submit(lambda: 1+1).result()"
  AttributeError: 'DaemonThreadPoolExecutor' object has no attribute '_initializer'

CI runs Python 3.13 (pyproject.toml: requires-python = ">=3.11,<3.14"),
so this never surfaced there — only on interpreters that have since
moved to 3.14.

Fix: branch on sys.version_info at import time and mirror whichever
shape of _worker/_adjust_thread_count the running interpreter actually
has (confirmed against 3.14.6's concurrent.futures.thread source: the
3.14 _adjust_thread_count calls _worker(weakref, self._create_worker_context(),
work_queue), the legacy shape calls _worker(weakref, work_queue,
initializer, initargs)). Both branches keep the two behavioral changes
that make this pool useful: daemon=True and no _threads_queues
registration.

Added a regression test (test_many_concurrent_submits_like_tool_executor)
reproducing the tool-executor's concurrent-submit shape; verified it
fails with the exact AttributeError above against the pre-fix code and
passes against the fix, on both Python 3.11 (uv venv, officially
supported) and 3.14.6 (local dev venv).

* fix(test): pin launchd domain in restart-recovery test to survive Aqua CI runners

test_launchd_restart_boots_out_stale_registration_before_bootstrap called
the real _launchd_domain() helper, which probes the live session type via
launchctl and returns gui/<uid> on an Aqua GUI runner instead of the
user/<uid> the test hardcodes into its plist path fixture. Pin the domain
the same way 9c155f3 pinned it for the refresh test: stub
_launchctl_session_managername to force headless, and make the launchctl
print probe (used by _launchd_domain_for_existing_job) fail so domain
resolution falls through to _launchd_domain() instead of matching an
already-loaded gui/<uid> job.

Unrelated to the daemon-pool py3.14 fix in this branch; found while
tracking down PR #53's slice-4/8 CI failure.

---------

Co-authored-by: Sahil (AI) <266772320+sahilm-ai@users.noreply.github.com>
sahilm-ti pushed a commit that referenced this pull request Jul 21, 2026
…cutor (#53)

* fix(tools): support CPython 3.14 WorkerContext in DaemonThreadPoolExecutor

CPython 3.14 replaced ThreadPoolExecutor's (initializer, initargs)
attribute pair + 4-arg _worker free function with a WorkerContext
object built via prepare_context()/_create_worker_context(). Our
DaemonThreadPoolExecutor._adjust_thread_count mirrored the pre-3.14
internals directly, so on 3.14 every worker spawn raised

  AttributeError: 'DaemonThreadPoolExecutor' object has no attribute
  '_initializer'

inside the worker thread, silently breaking every batch of 2+
concurrent tool calls (agent/tool_executor.py routes through this
pool). Reproduced empirically on the pre-fix code against the local
venv (Python 3.14.6, home = /opt/homebrew/opt/python@3.14):

  $ python3 -c "from tools.daemon_pool import DaemonThreadPoolExecutor; \
    DaemonThreadPoolExecutor(max_workers=2).submit(lambda: 1+1).result()"
  AttributeError: 'DaemonThreadPoolExecutor' object has no attribute '_initializer'

CI runs Python 3.13 (pyproject.toml: requires-python = ">=3.11,<3.14"),
so this never surfaced there — only on interpreters that have since
moved to 3.14.

Fix: branch on sys.version_info at import time and mirror whichever
shape of _worker/_adjust_thread_count the running interpreter actually
has (confirmed against 3.14.6's concurrent.futures.thread source: the
3.14 _adjust_thread_count calls _worker(weakref, self._create_worker_context(),
work_queue), the legacy shape calls _worker(weakref, work_queue,
initializer, initargs)). Both branches keep the two behavioral changes
that make this pool useful: daemon=True and no _threads_queues
registration.

Added a regression test (test_many_concurrent_submits_like_tool_executor)
reproducing the tool-executor's concurrent-submit shape; verified it
fails with the exact AttributeError above against the pre-fix code and
passes against the fix, on both Python 3.11 (uv venv, officially
supported) and 3.14.6 (local dev venv).

* fix(test): pin launchd domain in restart-recovery test to survive Aqua CI runners

test_launchd_restart_boots_out_stale_registration_before_bootstrap called
the real _launchd_domain() helper, which probes the live session type via
launchctl and returns gui/<uid> on an Aqua GUI runner instead of the
user/<uid> the test hardcodes into its plist path fixture. Pin the domain
the same way 9c155f3 pinned it for the refresh test: stub
_launchctl_session_managername to force headless, and make the launchctl
print probe (used by _launchd_domain_for_existing_job) fail so domain
resolution falls through to _launchd_domain() instead of matching an
already-loaded gui/<uid> job.

Unrelated to the daemon-pool py3.14 fix in this branch; found while
tracking down PR #53's slice-4/8 CI failure.

---------

Co-authored-by: Sahil (AI) <266772320+sahilm-ai@users.noreply.github.com>
sahilm-ti pushed a commit that referenced this pull request Jul 23, 2026
…cutor (#53)

* fix(tools): support CPython 3.14 WorkerContext in DaemonThreadPoolExecutor

CPython 3.14 replaced ThreadPoolExecutor's (initializer, initargs)
attribute pair + 4-arg _worker free function with a WorkerContext
object built via prepare_context()/_create_worker_context(). Our
DaemonThreadPoolExecutor._adjust_thread_count mirrored the pre-3.14
internals directly, so on 3.14 every worker spawn raised

  AttributeError: 'DaemonThreadPoolExecutor' object has no attribute
  '_initializer'

inside the worker thread, silently breaking every batch of 2+
concurrent tool calls (agent/tool_executor.py routes through this
pool). Reproduced empirically on the pre-fix code against the local
venv (Python 3.14.6, home = /opt/homebrew/opt/python@3.14):

  $ python3 -c "from tools.daemon_pool import DaemonThreadPoolExecutor; \
    DaemonThreadPoolExecutor(max_workers=2).submit(lambda: 1+1).result()"
  AttributeError: 'DaemonThreadPoolExecutor' object has no attribute '_initializer'

CI runs Python 3.13 (pyproject.toml: requires-python = ">=3.11,<3.14"),
so this never surfaced there — only on interpreters that have since
moved to 3.14.

Fix: branch on sys.version_info at import time and mirror whichever
shape of _worker/_adjust_thread_count the running interpreter actually
has (confirmed against 3.14.6's concurrent.futures.thread source: the
3.14 _adjust_thread_count calls _worker(weakref, self._create_worker_context(),
work_queue), the legacy shape calls _worker(weakref, work_queue,
initializer, initargs)). Both branches keep the two behavioral changes
that make this pool useful: daemon=True and no _threads_queues
registration.

Added a regression test (test_many_concurrent_submits_like_tool_executor)
reproducing the tool-executor's concurrent-submit shape; verified it
fails with the exact AttributeError above against the pre-fix code and
passes against the fix, on both Python 3.11 (uv venv, officially
supported) and 3.14.6 (local dev venv).

* fix(test): pin launchd domain in restart-recovery test to survive Aqua CI runners

test_launchd_restart_boots_out_stale_registration_before_bootstrap called
the real _launchd_domain() helper, which probes the live session type via
launchctl and returns gui/<uid> on an Aqua GUI runner instead of the
user/<uid> the test hardcodes into its plist path fixture. Pin the domain
the same way 9c155f3 pinned it for the refresh test: stub
_launchctl_session_managername to force headless, and make the launchctl
print probe (used by _launchd_domain_for_existing_job) fail so domain
resolution falls through to _launchd_domain() instead of matching an
already-loaded gui/<uid> job.

Unrelated to the daemon-pool py3.14 fix in this branch; found while
tracking down PR #53's slice-4/8 CI failure.

---------

Co-authored-by: Sahil (AI) <266772320+sahilm-ai@users.noreply.github.com>
sahilm-ti pushed a commit that referenced this pull request Jul 28, 2026
…cutor (#53)

* fix(tools): support CPython 3.14 WorkerContext in DaemonThreadPoolExecutor

CPython 3.14 replaced ThreadPoolExecutor's (initializer, initargs)
attribute pair + 4-arg _worker free function with a WorkerContext
object built via prepare_context()/_create_worker_context(). Our
DaemonThreadPoolExecutor._adjust_thread_count mirrored the pre-3.14
internals directly, so on 3.14 every worker spawn raised

  AttributeError: 'DaemonThreadPoolExecutor' object has no attribute
  '_initializer'

inside the worker thread, silently breaking every batch of 2+
concurrent tool calls (agent/tool_executor.py routes through this
pool). Reproduced empirically on the pre-fix code against the local
venv (Python 3.14.6, home = /opt/homebrew/opt/python@3.14):

  $ python3 -c "from tools.daemon_pool import DaemonThreadPoolExecutor; \
    DaemonThreadPoolExecutor(max_workers=2).submit(lambda: 1+1).result()"
  AttributeError: 'DaemonThreadPoolExecutor' object has no attribute '_initializer'

CI runs Python 3.13 (pyproject.toml: requires-python = ">=3.11,<3.14"),
so this never surfaced there — only on interpreters that have since
moved to 3.14.

Fix: branch on sys.version_info at import time and mirror whichever
shape of _worker/_adjust_thread_count the running interpreter actually
has (confirmed against 3.14.6's concurrent.futures.thread source: the
3.14 _adjust_thread_count calls _worker(weakref, self._create_worker_context(),
work_queue), the legacy shape calls _worker(weakref, work_queue,
initializer, initargs)). Both branches keep the two behavioral changes
that make this pool useful: daemon=True and no _threads_queues
registration.

Added a regression test (test_many_concurrent_submits_like_tool_executor)
reproducing the tool-executor's concurrent-submit shape; verified it
fails with the exact AttributeError above against the pre-fix code and
passes against the fix, on both Python 3.11 (uv venv, officially
supported) and 3.14.6 (local dev venv).

* fix(test): pin launchd domain in restart-recovery test to survive Aqua CI runners

test_launchd_restart_boots_out_stale_registration_before_bootstrap called
the real _launchd_domain() helper, which probes the live session type via
launchctl and returns gui/<uid> on an Aqua GUI runner instead of the
user/<uid> the test hardcodes into its plist path fixture. Pin the domain
the same way 9c155f3 pinned it for the refresh test: stub
_launchctl_session_managername to force headless, and make the launchctl
print probe (used by _launchd_domain_for_existing_job) fail so domain
resolution falls through to _launchd_domain() instead of matching an
already-loaded gui/<uid> job.

Unrelated to the daemon-pool py3.14 fix in this branch; found while
tracking down PR #53's slice-4/8 CI failure.

---------

Co-authored-by: Sahil (AI) <266772320+sahilm-ai@users.noreply.github.com>
sahilm-ti pushed a commit that referenced this pull request Aug 24, 2026
…cutor (#53)

* fix(tools): support CPython 3.14 WorkerContext in DaemonThreadPoolExecutor

CPython 3.14 replaced ThreadPoolExecutor's (initializer, initargs)
attribute pair + 4-arg _worker free function with a WorkerContext
object built via prepare_context()/_create_worker_context(). Our
DaemonThreadPoolExecutor._adjust_thread_count mirrored the pre-3.14
internals directly, so on 3.14 every worker spawn raised

  AttributeError: 'DaemonThreadPoolExecutor' object has no attribute
  '_initializer'

inside the worker thread, silently breaking every batch of 2+
concurrent tool calls (agent/tool_executor.py routes through this
pool). Reproduced empirically on the pre-fix code against the local
venv (Python 3.14.6, home = /opt/homebrew/opt/python@3.14):

  $ python3 -c "from tools.daemon_pool import DaemonThreadPoolExecutor; \
    DaemonThreadPoolExecutor(max_workers=2).submit(lambda: 1+1).result()"
  AttributeError: 'DaemonThreadPoolExecutor' object has no attribute '_initializer'

CI runs Python 3.13 (pyproject.toml: requires-python = ">=3.11,<3.14"),
so this never surfaced there — only on interpreters that have since
moved to 3.14.

Fix: branch on sys.version_info at import time and mirror whichever
shape of _worker/_adjust_thread_count the running interpreter actually
has (confirmed against 3.14.6's concurrent.futures.thread source: the
3.14 _adjust_thread_count calls _worker(weakref, self._create_worker_context(),
work_queue), the legacy shape calls _worker(weakref, work_queue,
initializer, initargs)). Both branches keep the two behavioral changes
that make this pool useful: daemon=True and no _threads_queues
registration.

Added a regression test (test_many_concurrent_submits_like_tool_executor)
reproducing the tool-executor's concurrent-submit shape; verified it
fails with the exact AttributeError above against the pre-fix code and
passes against the fix, on both Python 3.11 (uv venv, officially
supported) and 3.14.6 (local dev venv).

* fix(test): pin launchd domain in restart-recovery test to survive Aqua CI runners

test_launchd_restart_boots_out_stale_registration_before_bootstrap called
the real _launchd_domain() helper, which probes the live session type via
launchctl and returns gui/<uid> on an Aqua GUI runner instead of the
user/<uid> the test hardcodes into its plist path fixture. Pin the domain
the same way 9c155f3 pinned it for the refresh test: stub
_launchctl_session_managername to force headless, and make the launchctl
print probe (used by _launchd_domain_for_existing_job) fail so domain
resolution falls through to _launchd_domain() instead of matching an
already-loaded gui/<uid> job.

Unrelated to the daemon-pool py3.14 fix in this branch; found while
tracking down PR #53's slice-4/8 CI failure.

---------

Co-authored-by: Sahil (AI) <266772320+sahilm-ai@users.noreply.github.com>
sahilm-ti pushed a commit that referenced this pull request Sep 2, 2026
…cutor (#53)

* fix(tools): support CPython 3.14 WorkerContext in DaemonThreadPoolExecutor

CPython 3.14 replaced ThreadPoolExecutor's (initializer, initargs)
attribute pair + 4-arg _worker free function with a WorkerContext
object built via prepare_context()/_create_worker_context(). Our
DaemonThreadPoolExecutor._adjust_thread_count mirrored the pre-3.14
internals directly, so on 3.14 every worker spawn raised

  AttributeError: 'DaemonThreadPoolExecutor' object has no attribute
  '_initializer'

inside the worker thread, silently breaking every batch of 2+
concurrent tool calls (agent/tool_executor.py routes through this
pool). Reproduced empirically on the pre-fix code against the local
venv (Python 3.14.6, home = /opt/homebrew/opt/python@3.14):

  $ python3 -c "from tools.daemon_pool import DaemonThreadPoolExecutor; \
    DaemonThreadPoolExecutor(max_workers=2).submit(lambda: 1+1).result()"
  AttributeError: 'DaemonThreadPoolExecutor' object has no attribute '_initializer'

CI runs Python 3.13 (pyproject.toml: requires-python = ">=3.11,<3.14"),
so this never surfaced there — only on interpreters that have since
moved to 3.14.

Fix: branch on sys.version_info at import time and mirror whichever
shape of _worker/_adjust_thread_count the running interpreter actually
has (confirmed against 3.14.6's concurrent.futures.thread source: the
3.14 _adjust_thread_count calls _worker(weakref, self._create_worker_context(),
work_queue), the legacy shape calls _worker(weakref, work_queue,
initializer, initargs)). Both branches keep the two behavioral changes
that make this pool useful: daemon=True and no _threads_queues
registration.

Added a regression test (test_many_concurrent_submits_like_tool_executor)
reproducing the tool-executor's concurrent-submit shape; verified it
fails with the exact AttributeError above against the pre-fix code and
passes against the fix, on both Python 3.11 (uv venv, officially
supported) and 3.14.6 (local dev venv).

* fix(test): pin launchd domain in restart-recovery test to survive Aqua CI runners

test_launchd_restart_boots_out_stale_registration_before_bootstrap called
the real _launchd_domain() helper, which probes the live session type via
launchctl and returns gui/<uid> on an Aqua GUI runner instead of the
user/<uid> the test hardcodes into its plist path fixture. Pin the domain
the same way 9c155f3 pinned it for the refresh test: stub
_launchctl_session_managername to force headless, and make the launchctl
print probe (used by _launchd_domain_for_existing_job) fail so domain
resolution falls through to _launchd_domain() instead of matching an
already-loaded gui/<uid> job.

Unrelated to the daemon-pool py3.14 fix in this branch; found while
tracking down PR #53's slice-4/8 CI failure.

---------

Co-authored-by: Sahil (AI) <266772320+sahilm-ai@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant