Skip to content

fix: Python 3.14 compatibility (daemon_pool worker signature, gateway liveness argv, dotenv reload race) - #61224

Open
slow4cyl wants to merge 4 commits into
NousResearch:mainfrom
slow4cyl:fix/py314-compat
Open

fix: Python 3.14 compatibility (daemon_pool worker signature, gateway liveness argv, dotenv reload race)#61224
slow4cyl wants to merge 4 commits into
NousResearch:mainfrom
slow4cyl:fix/py314-compat

Conversation

@slow4cyl

@slow4cyl slow4cyl commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Three small fixes that together make the agent run correctly on Python 3.14. Each is its own commit:


fix(daemon_pool): Python 3.14 ThreadPoolExecutor._worker signature compat

Python 3.14 changed ThreadPoolExecutor._worker to (executor_ref, ctx,
work_queue), folding initializer/initargs into a worker context object.
The subclass still passed the old 4-tuple, so every pool thread died at
spawn: on 3.14 the kanban dispatcher spawned zero workers. Branch on the
presence of _create_worker_context so 3.8-3.13 keep the old path.

fix(gateway): status liveness check tolerates setproctitle-stripped argv

main.py calls setproctitle("hermes"), erasing the gateway subcommand
from /proc cmdline. The liveness check then failed its "looks like a
gateway" test and reported a healthy systemd gateway as OFFLINE. When
argv carries no gateway subcommand but PID + start_time match the
persisted record, defer to the recorded gateway identity.

fix(env_loader): retry dotenv reload on transient KeyError

python-dotenv's env.update(os.environ) can race another thread mutating
os.environ (the in-gateway kanban decomposer pins/unpins env vars per
board), raising a transient KeyError that crashed in-gateway cron jobs.
Retry once on that specific failure.


🤖 Generated with Claude Code

@alt-glitch alt-glitch added type/bug Something isn't working comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/gateway Gateway runner, session dispatch, delivery comp/cli CLI entry point, hermes_cli/, setup wizard P3 Low — cosmetic, nice to have labels Jul 9, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.

Related, not a duplicate. The tools/daemon_pool.py half of this omnibus overlaps the earliest-open Python 3.14 _worker-signature fix #57459 (and its closed siblings), but this PR bundles two additional novel 3.14 fixes not covered there: the gateway/status.py liveness check tolerating setproctitle-stripped argv, and the hermes_cli/env_loader.py dotenv-reload retry on transient KeyError. Flagging the daemon_pool overlap so a reviewer can decide whether to take the daemon_pool half from #57459 and the other two from here, or land this whole PR.

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the focused compatibility work. The daemon-pool branch matches CPython 3.14's _create_worker_context() / three-argument _worker shape, and current main still has the pre-3.14 call in tools/daemon_pool.py:52-60.

Problems

  • The PR cannot yet make Hermes runnable as a supported Python 3.14 installation: pyproject.toml:13-20 deliberately caps requires-python below 3.14 because required Rust-backed dependencies lack cp314 wheels.
  • hermes_cli/env_loader.py:162 invokes the Latin-1 fallback outside the new KeyError retry handler. A transient KeyError in that fallback escapes rather than retrying.
  • No regression tests are included for the three changed paths; relevant suites already exist at tests/tools/test_daemon_pool.py, tests/gateway/test_status.py, and tests/hermes_cli/test_env_loader.py.

Suggested changes

  • Put both encoding paths inside the KeyError retry loop, then add focused regressions for the 3.14 worker signature, bare hermes gateway argv, and KeyError on both encoding paths.
  • Re-scope the Python 3.14 claim or separately resolve the packaging constraint in pyproject.toml:13-20.

Automated hermes-sweeper review.

Comment thread hermes_cli/env_loader.py
# gateway/kanban_watchers.py), so dict.update can KeyError on a key that
# vanished between keys() and __getitem__ (crashed cron jobs, #external-probe).
# The window is microseconds and the failing load applied nothing (the error
# is in .dict() before set_as_environment_variables), so retrying is safe.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This Latin-1 fallback is outside the new except KeyError block. If the same concurrent-environment KeyError occurs after UTF-8 decoding fails, it escapes without a retry; wrap the encoding fallback inside the retry attempt.

@teknium1 teknium1 added sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-platform-windows Sweeper risk: may break or behave differently on native Windows sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform labels Jul 10, 2026
slow4cyl and others added 4 commits July 10, 2026 11:33
…mpat

Python 3.14 changed ThreadPoolExecutor._worker to (executor_ref, ctx,
work_queue), folding initializer/initargs into a worker context object.
The subclass still passed the old 4-tuple, so every pool thread died at
spawn: on 3.14 the kanban dispatcher spawned zero workers. Branch on the
presence of _create_worker_context so 3.8-3.13 keep the old path.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
main.py calls setproctitle("hermes"), erasing the gateway subcommand
from /proc cmdline. The liveness check then failed its "looks like a
gateway" test and reported a healthy systemd gateway as OFFLINE. When
argv carries no gateway subcommand but PID + start_time match the
persisted record, defer to the recorded gateway identity.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
python-dotenv's env.update(os.environ) can race another thread mutating
os.environ (the in-gateway kanban decomposer pins/unpins env vars per
board), raising a transient KeyError that crashed in-gateway cron jobs.
Retry once on that specific failure.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ack, regression tests

Per review (scope: forward-compat for Python 3.14, NOT a support claim —
pyproject requires-python stays capped below 3.14 until the Rust-backed
deps ship cp314 wheels; this branch never touched it):

- env_loader: hoist the latin-1 fallback into the KeyError retry loop —
  a transient KeyError raised inside the fallback previously escaped
  instead of retrying (review finding at env_loader.py:162). Both
  encoding paths now run inside the 3-attempt loop.
- gateway/status: narrow the setproctitle fallback. Deferring to the
  record whenever the argv lacked a gateway subcommand regressed
  test_runtime_status_running_pid_rejects_stale_record_for_supervisor_pid
  (a readable s6-supervise cmdline on a matching-start-time PID was
  reported running). New _cmdline_is_bare_hermes_proctitle() accepts
  exactly the stripped setproctitle("hermes") shape — one token,
  basename hermes/hermes.exe; every other readable cmdline keeps the
  strict live-argv judgment.
- daemon_pool: extract _worker_args() so the version-guarded branch is
  directly unit-testable; behavior unchanged (hasattr probe on the
  instance — on 3.14 _create_worker_context is set in __init__).
- Tests (all three suites named by the review):
  * daemon_pool: probe-matches-interpreter guard, both _worker arg
    shapes pinned via stand-in executors (covers the 3.14 branch on
    <=3.13 CI and the legacy branch on 3.14), live-pool spawn guard.
  * status: bare-hermes proctitle reported running; stripped-proctitle
    fallback still validates the record; non-proctitle readable
    cmdlines stay strict; direct proctitle matcher tests.
  * env_loader: KeyError retry on the utf-8 path, on the latin-1
    fallback (the review's exact case), persistent-KeyError re-raise
    after 3 attempts, and a real non-UTF-8 file loading via latin-1.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@slow4cyl

Copy link
Copy Markdown
Contributor Author

Re-scoped as requested: this is forward-compatibility for 3.14 — every 3.14 branch is feature-detected and a no-op on ≤3.13, and requires-python was never bumped here (stays capped until the Rust-backed deps ship cp314 wheels). env_loader now runs both encoding paths inside the KeyError retry loop, with a regression pinning the latin-1-fallback case you flagged. While adding the requested status tests I found the record-fallback was too broad — a readable s6-supervise gateway-coder cmdline on a matching-start-time PID regressed test_runtime_status_running_pid_rejects_stale_record_for_supervisor_pid — so it's narrowed to exactly the stripped setproctitle("hermes") shape (_cmdline_is_bare_hermes_proctitle); all other readable cmdlines keep strict judgment. Added regressions in all three suites you named, including stand-in executors that pin both _worker arg shapes on any interpreter and an interpreter-probe guard so a future stdlib reshape fails loudly. On a 3.14 interpreter the four pre-existing daemon_pool tests fail on main and pass here; no other failure-set changes. Rebased onto current main.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/cli CLI entry point, hermes_cli/, setup wizard comp/gateway Gateway runner, session dispatch, delivery P3 Low — cosmetic, nice to have sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-platform-windows Sweeper risk: may break or behave differently on native Windows type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants