[MGiotto]

Summary

• Route Kanban worker starts, gateway update/restart, and CLI self-relaunch through the current installation's durable launcher or an isolated -I bootstrap anchored to the imported source tree.
• Remove CWD-relative and ambient-PATH fallback from CLI self-relaunch; keep explicit HERMES_BIN for dispatcher workers and consult PATH only if the install-bound resolver cannot be built.
• Add isolated-import, actual CLI launch, worker argv, and real worker retry coverage. The Codex E2E fixture verifies a first-use 429 stays on openai-codex, the reclaimed worker retries the same model/provider, no default-provider request occurs, and state.db records billing_provider=openai-codex.

Verification

• INVOCATION_ID= /home/hermes/.hermes/installs/e7f97e460043e6c5/test-environment/gen-ad0728e4804d4b94a46396747fc7edf7/venv/bin/python -m pytest tests/gateway/test_update_command.py tests/hermes_cli/test_relaunch.py tests/hermes_cli/test_kanban_db.py tests/hermes_cli/test_kanban_worker_spawn_toolsets.py tests/e2e/core/kanban/test_kanban_rate_limit_review.py tests/e2e/core/kanban/test_kanban_dispatcher_restart.py tests/e2e/core/kanban/test_kanban_worker_relaunch_provider.py -q — 93 passed, 5 skipped, 1 xfailed.
• .../venv/bin/python -m ruff check over all nine changed Python files — passed.
• git diff --cached --check and git diff origin/main...HEAD --check — passed.
• The new worker E2E uses a loopback fake Codex Responses server and a temporary board/home only; no external inference or OAuth endpoint is contacted.

Environment note / limitation

scripts/run_tests.sh could not finish its activation step here (activate: no bootstrap Python found; run setup-hermes.sh). The focused suite was run directly with the PM test-environment interpreter above; INVOCATION_ID= isolates unit tests from this host's systemd service context. For upstream PR CI, the three workflows have action_required with zero jobs/check runs reported on head 4bee417d25c9d207a61f8719363b79ff6c576c02; a maintainer must approve the cross-fork workflow run.

No merge, deployment, gateway restart, or EA1 task unblock/retry was performed.

[Alexandre78500]

Independent reproduction + a note on scope (macOS arm64, PM store Python 3.14.7, multi-profile gateways).

Hit this live today: every kanban worker died with No module named 'hermes_cli'. One detail worth knowing when verifying: with several profile gateways, the dispatcher lock may be held by a non-default profile's gateway, so restarting only the default gateway after applying the fix changes nothing — the gateway holding kanban/.dispatcher.lock must be restarted.

Same bug class, sibling sites not covered by this PR (nor #122641 / #122420) — each still builds a bare [sys.executable, "-m", "hermes_cli.main"] on current main, so each breaks the same way when the parent runs isolated on the store Python:

• cron/scheduler_delivery.py — Bot Chat delivery subprocess
• hermes_cli/update_restart_recovery.py — _run_profile_restart
• hermes_cli/main_dashboard.py — named-profile re-exec (also needs os.execvpe(argv[0], …) since argv[0] is no longer always sys.executable)
• gateway/slash_commands.py — Windows detached /update helper inner command

What I ran locally, for reference: one helper in hermes_cli/_launchers.py used by all 7 sites —

def current_install_hermes_argv(args=(), *, module="hermes_cli.main") -> list[str]:
    if not sys.flags.isolated:
        return [sys.executable, "-m", module, *args]
    return runtime_command(Path(__file__).resolve().parents[1], args, module=module,
                           python=sys.executable)

Gating on sys.flags.isolated rather than on "is store Python" keeps the module form (and PYTHONPATH) for venv/Nix interpreters, so Nix extraPythonPackages plugins still reach children that ignore HERMES_BIN; #111569 is preserved (argv bound to the running install, PATH never consulted first).

Regression test: E2E — ask the real .hermes/bin/hermes --print-runtime-command for the store interpreter, start a probe in the isolated bootstrap form, resolve the child argv through each production resolver, then run <argv> --version from a temp cwd with no PYTHONPATH and PATH=/usr/bin:/bin. Red on the old code (exact ModuleNotFoundError), green with the helper; kanban / relaunch / update-restart-recovery / cron bot-chat delivery / restart-drain suites green. After deploying + restarting the lock-holding gateway, a real worker ran to completion.

Happy to open a follow-up for the 4 remaining sites once this lands, or you can fold them in here — whichever maintainers prefer.

[Enough1122]

AI code review — automated review for reference; please use your judgment.

三处入口现在共同依赖一个从不检查 launcher 存在性的分支

installation_command() 只问 resolve_store_python(root) is None(_launchers.py:74),从不问 .hermes/bin/hermes 是否在磁盘上 —— rb/122597 blocker 2 / rb/123041 blocker 1 那一格没修正,反被三处入口接上。

Blocker 1 — .cmd-only 或缺 launcher 的安装 100% spawn 失败

_launchers.py:76-77 无条件返回 str(root/".hermes"/"bin"/"hermes")(无扩展名)。真 spawn 实测:

磁盘上的 launcher	argv[0]	真 spawn
hermes.exe(真 distlib PE)	无扩展名	rc=0(CreateProcess 补 .exe)
只发布 hermes.cmd	无扩展名	FileNotFoundError WinError 2
什么都没有,store python 在	无扩展名	WinError 2

坏的不是"无扩展名"本身 —— .exe 那格能跑;坏的是 .cmd 这一格 CreateProcess 不补,而 _launchers.py:277 正是 stage_launcher() 在 distlib 不可用时的活路径。

这是相对 base 的回归:base 的 [sys.executable, "-m", "hermes_cli.main"] 在该状态照常能跑。派发侧 Popen 的 FileNotFoundError 不是 RestartSafeScopeUnavailable(process_registry.py:328,RuntimeError 子类),故 kanban_db_dispatch.py:2094 的 infrastructure=False → consecutive_failures 照扣 → DEFAULT_FAILURE_LIMIT=2 两下 park。建议:按 hermes.exe → hermes.cmd → hermes 探测存在性,都没有就退回 runtime_command。

Blocker 2 — gateway 侧 fail-open,#111569 回归

gateway/run.py:_resolve_hermes_bin() 保留 except Exception: shutil.which("hermes")。实测:让 current_installation_command 抛错 + PATH 上有 /tmp/ATTACKER/hermes → 返回它。它内部 store_root() → pm/environments.py:109 的 json.loads(manifest.json) 无 try,corrupt manifest 直接冒到 /update。

对比 kanban_db_dispatch.py:2540 同样 raise RuntimeError(fail-closed)—— 同一 commit 里两个同批入口给了相反答案。

非阻塞

stamp 向上继承:pm/environments.py:117-125 沿 root.parents 找第一个 stamp,实测无 stamp 的子目录继承了父目录的 store python,argv0 指向不存在的 worktree/.hermes/bin/hermes → WinError 2(同 rb/123421)。

测试:Windows 基线 3 failed, 37 passed;建 base worktree 对跑,三个 base 上同样红(断言写死 POSIX 路径),非本 PR 引入。

三次 mutation:M1 = 给 installation_command:76 加 launcher 存在性检查(即修好 blocker 1)→ 结果与基线逐字一致,Windows 套件零变化。 M2、M3 被捕捉,但捕捉的是路径优先级;"能不能 exec"一层无人看守。

⚠️ 真 spawn 的两个测试恰好都落在 launcher 分支之外:本机 HERMES_RUNTIME_DIR 未设 → resolve_store_python(repo)=None → 走 runtime_command()。.hermes/bin/hermes 分支零真 spawn 覆盖,正是 blocker 1 所在处。新加 e2e 有真 spawn 但 skipif(not linux);toolsets 只测字段。

不会无限重试,但上限=事故:spawn_failed 且非 infrastructure 时 infrastructure_cooldown(:1534-1538)不生效 → 零退避、2 下 park。