Skip to content

fix(cron): read the committed generation for Windows cron scripts - #123668

Closed
Halldrix wants to merge 9 commits into
NousResearch:mainfrom
Halldrix:fix/cron-windows-venv-abi-123650
Closed

Halldrix wants to merge 9 commits into
NousResearch:mainfrom
Halldrix:fix/cron-windows-venv-abi-123650

Conversation

@Halldrix

@Halldrix Halldrix commented Sep 26, 2026 •

Copy link
Copy Markdown

What does this PR do?

Windows cron scripts could be spawned onto a venv built for a different interpreter than the one running them, killing every script with No module named 'pydantic_core._pydantic_core'.

cron/scheduler_script.py::_windows_cron_python_invocation answers "which interpreter and which site-packages?" for a cron script child. On a PM-managed install it read the dependency tree through pm.environments.selected_venv, which falls back to base_venv and answers the leftover pre-PM <root>/venv when no generation is recorded. That tree belongs to whichever interpreter created it — typically CPython 3.11 while the managed store Python is 3.14 — so overlaying it on PYTHONPATH puts a cp311 pydantic_core in front of the running interpreter. Pure-Python imports succeed; the first compiled extension does not:

ModuleNotFoundError: No module named 'pydantic_core._pydantic_core'

committed_venv is the resolver that never answers with the in-tree venv. With a generation committed, the child gets that generation's site-packages. With none — or a record too corrupt to read — the call falls through to the uv overlay below it, so the handed venv keeps its own interpreter and its own packages, exactly as on main.

This is the cron sibling of the gateway crash in #122183 (and the family reported again in #123650): gateway/run.py::_ensure_windows_gateway_venv_imports made the same mistake with the same resolver, and no PR covers the cron side of it.

Related Issue

Fixes the cron side of #122183. The gateway half is tracked by #122330, #122333, #123149, #122739 and #123206; this PR deliberately does not touch gateway/run.py or overlap any of them.

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✅ Tests (adding or improving test coverage)

Changes Made

  • cron/scheduler_script.py — _windows_cron_python_invocation: resolve the managed install's dependency tree with committed_venv instead of selected_venv. With a generation committed, return the store Python with that generation's site-packages. With none, or an unreadable record (which raises rather than answering None), fall through to the uv overlay. The PM read is wrapped in the repo's idiom (except (OSError, RuntimeError, ValueError) — environments_adopt.py:30, install.py:524), not bare Exception, and payload_venv's fail-closed "payload environment escapes its root" (pm/environments.py:92) is re-raised rather than downgraded to a warning.
  • tools/environments/local_pythonpath.py — _validated_runtime_venv: the same wrap on the sibling read. Its runtime_facts_path().is_file() gate suppressed the fallback, never the raise, so a malformed facts.json still raised inside _finalize_child_env — the guard path every spawn surface shares.
  • tests/cron/test_cron_windows_venv_abi.py (new, 3 tests) — gated pytest.mark.platforms("windows") so the Windows CI lane actually imports the file. The committed generation wins over the in-tree venv; no-generation and unreadable-record both land on the handed venv; and the fall-through's stdlib arm (a venv with no uv key) keeps its own python.exe.
  • tests/tools/test_local_env_blocklist.py (+2 tests) — a malformed PM record degrades to no runtime venv instead of raising; payload_venv's escape check still propagates.

How to Test

Verified on a real Windows 11 host — the change is Windows-only (_windows_cron_python_invocation early-returns elsewhere), so a Linux run proves only that the file is correctly gated.

  1. GREEN on the branch head (f29885969c), the 3 new + 2 added tests: 3 passed for the new cron file, 2 passed for the blocklist additions.
  2. RED with the sibling guard removed (local_pythonpath.py back at 30c649f16b): the malformed-record test fails with the raw RuntimeError from _recorded_venv — 1 failed.
  3. RED with only the escape carve-out removed (guard intact, the "escapes its root" → raise line deleted): the escape-propagation test fails alone — 1 failed. This is the isolation that shows the narrow catch is load-bearing, not incidental.
  4. Neighbour suites: tests/cron/test_cron_script.py 31 passed / 1 skipped; tests/tools/test_local_env_blocklist.py 74 passed, 1 failed — that one failure, test_runtime_provenance_is_independent_of_aliases_and_virtual_env[True], reproduces identically on pristine main in a separate worktree, so it is pre-existing and not from this diff.
  5. Windows lane picks the file up — python scripts/ci/list_os_marked_tests.py windows . lists tests/cron/test_cron_windows_venv_abi.py (it did not with the original skipif gate).
  6. The reproducer from the review thread, run against this head — a handed uv venv, a managed store Python, nothing committed:
interpreter : <tmp>\uvbase\python.exe          <- the handed venv's base, not the store
PYTHONPATH  : <repo>;<tmp>\ownvenv\Lib\site-packages
VIRTUAL_ENV : <tmp>\ownvenv
bootstrap   : ['-c', "...site.addsitedir('<tmp>\ownvenv\Lib\site-packages')..."]

No WARNING ... venv site-packages not found line, and the bootstrap processes .pth files. On the pre-fix head the same probe returned the bare store Python, a repo-only PYTHONPATH, and warned on every spawn.
7. Host left clean — worktree removed, live repo back on main with an empty git status.

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
  • I've searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix/feature (no unrelated commits)
  • I've run the relevant tests and they pass (see How to Test for the W11 receipts and the pre-existing failure reproduced on main)
  • I've added tests for my changes (required for bug fixes)
  • I've tested on my platform: Windows 11 (real host, GREEN + two isolated RED sabotages + neighbour suites)

Documentation & Housekeeping

  • I've updated relevant documentation — the call-site comments carry the WHY — or N/A
  • I've updated cli-config.yaml.example if I added/changed config keys — or N/A (no config keys)
  • I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — or N/A
  • I've considered cross-platform impact (Windows, macOS) per the compatibility guide — the cron change is Windows-gated; local_pythonpath is cross-platform, so the catch uses the repo's shared PM-read idiom
  • I've updated tool descriptions/schemas if I changed tool behavior — or N/A

Notes for reviewers

Why this is not a sixth PR on _ensure_windows_gateway_venv_imports. That function already has five open PRs (#122330, #122333, #123149, #122739, #123206) and its canonical issue #122183 carries ~10 independent reproductions. This one covers a different file that none of them touch.

Overlap with #123479. That PR also edits cron/scheduler_script.py, but a different defect: its POSIX branch has no dependency environment at all. Its hunk is the sys.platform != "win32" early-return; this one is the managed-install block below it. Neither diff mentions the other's symbols, so they should merge independently — but note #123479 keeps selected_venv, so merging it after this one reintroduces the in-tree fallback.

The class, and where it actually ends. _windows_cron_python_invocation and local_pythonpath._validated_runtime_venv were the two callers of selected_venv that fed a child's environment. Both now read through a wrapped committed_venv / degrade-on-unreadable-record. The remaining callers — hermes_cli/_early_recovery.py:527, pm/extras.py:226, pm/environments_adopt.py:29, hermes_cli/runtime_state.py:184, pm/install.py:523 — use the result for identity comparison or guarded lookup and never hand it to a child. (An earlier draft justified that with "every other caller is guarded by runtime_facts_path().is_file()"; that was wrong as stated for three of them. The conclusion holds for the better reason: none of them composes a child env from it.)

hermes_cli/gateway.py::_pm_runtime_venv_dir also reads selected_venv, but it has no callers — dead code, left alone deliberately.

Screenshots / Logs

W11 transcript (head f29885969c):

HEAD: f29885969c (expect f29885969c)
MARKER sibling guard: True
===== GREEN (pushed head) =====
  3 passed, 80 deselected in 5.04s

[RED1] sibling guard gone: True
===== RED1 (sibling guard removed) =====
  1 failed, 79 deselected in 2.94s

[RED2] only the escape carve-out removed: True
===== RED2 (only the escape carve-out removed) =====
  1 failed, 79 deselected in 2.90s

restore: carve-out back: True
dirty: ''
RESULT green=0 red1=1 red2=1

…usResearch#122183)

_windows_cron_python_invocation selected the dependency tree with
selected_venv, which falls back to base_venv and answers the leftover
pre-PM <root>/venv when no generation is recorded. That tree belongs to
whichever interpreter created it, so on a PM-managed install the managed
store Python 3.14 got a cp311 site-packages on PYTHONPATH and every cron
script died with "No module named 'pydantic_core._pydantic_core'" -- the
cron sibling of the gateway crash in NousResearch#122183, which janviernine flagged
there and left out of scope.

committed_venv never answers with the in-tree venv. With nothing committed
this install provisioned no tree, so overlay none and let the child keep
the interpreter it was handed rather than borrowing a foreign ABI.

Every other selected_venv caller (tools/environments/local_pythonpath,
hermes_cli/_early_recovery, pm/extras, pm/environments_adopt) already gates
on runtime_facts_path().is_file(); this was the only unguarded one.
… tree (NousResearch#122183)

Three invariants over _windows_cron_python_invocation: a managed install
with no committed generation never overlays the leftover pre-PM venv, a
committed generation is the tree that gets overlaid, and a non-managed
install keeps its uv base-interpreter overlay unchanged.
@alt-glitch alt-glitch added type/bug Something isn't working P1 High — major feature broken, no workaround comp/cron Cron scheduler and job management platform/windows area/install-update Installer, updater, packaging, wheels, doctor sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-platform-windows Sweeper risk: may break or behave differently on native Windows labels Sep 26, 2026
@janviernine

Copy link
Copy Markdown

Beautifully scoped — this is exactly the sibling I flagged on #122183, and the Green/Red/Control test story (2 fail with the fix reverted, legacy stays green) is the right evidence. One residual, same failure-direction class as the two rounds on #122330:

committed_venv(repo) can raise, not just return None: _recorded_venv raises RuntimeError on a corrupt/unparseable facts.json (pm/environments.py:183-192), and dependency_site(environment) raises if the committed tree vanished. Both propagate straight out of _windows_cron_python_invocation — _script_argv calls it unguarded (scheduler_script.py:331), so a corrupt record turns every cron script into a traceback instead of a run. Narrow window, silent-ish direction: jobs fail at spawn with an environment error rather than the familiar pydantic_core one, and nothing in the log points at the record.

The fail-closed shape matches this PR's own None arm: wrap the resolve in try/except, overlay repo-only, and let the child keep the interpreter it was handed:

try:
    environment = committed_venv(repo)
except Exception:
    environment = None

Non-blocking — the reported defect is fixed as is. Flagging it because the comment asserts the install provisioned no tree, and on the raise path the code never gets to assert anything.

— reviewed with KBG Bot MKIV, courtesy of the KBGville Discord community.

…dable (NousResearch#122183)

committed_venv raises - it does not answer None - on an unparseable or
out-of-tree facts.json (pm/environments.py:179-193), and site_packages
raises when the committed tree vanished. _script_argv calls this
unguarded, so a corrupt record turned every cron script into a
spawn-time traceback with nothing in the log pointing at the record.

Catch it and take the same shape as the None arm: repo-only overlay, the
child keeps the interpreter it was handed, WARNING logged. Degraded
rather than dead. The pre-existing selected_venv call could raise the
same way, so this is not a regression introduced here - but the comment
asserted an invariant that the raise path never reached.
…Research#122183)

A committed_venv that raises must still hand the store interpreter to the
child and overlay repo-only, never the stale in-tree tree.
…NousResearch#122183)

The overlay carries Path(__file__).parents[1] - the real checkout - so the
tmp_path repo built for the stale venv never appears in it. Caught by the
W11 run: the assertion failed against correct code.
@Halldrix

Copy link
Copy Markdown
Author

Good catch — verified at the source and taken.

committed_venv raises rather than answering None: _recorded_venv raises RuntimeError on an unparseable or out-of-tree facts.json (pm/environments.py:179-193), and site_packages raises when the committed tree vanished. _script_argv calls this unguarded (scheduler_script.py:331), so a corrupt record turned every cron script into a spawn-time traceback — and my comment claimed "with no generation committed this install provisioned no tree", which the raise path never reaches.

One correction to the framing, though: this is not a regression introduced here. The pre-existing selected_venv call sits directly above the same raise, so a corrupt record could abort the spawn before this PR too. What did change is that my comment asserted an invariant the raise path never satisfied. Pushed as 43751c7392, with 06a11f1edb adding the test that pins it.

The guard takes the same shape as the None arm — repo-only overlay, the child keeps the interpreter it was handed, WARNING logged — so a bad record degrades a job instead of killing it:

try:
    environment = committed_venv(repo)
except Exception:
    logger.warning(
        "Windows cron script: could not resolve the committed dependency environment "
        "for %s; running without a dependency overlay", repo, exc_info=True)
    environment = None

Re-verified on a real Windows 11 host, now with both sabotages doing real work (the first harness re-added the worktree between phases and silently undid them, so the earlier RED run was vacuous):

phase result
GREEN on b86dffade 4 passed
RED1 — prod reverted to main's d0288be5b 3 failed, 1 passed
RED2 — only the try/except reverted 1 failed, 3 passed

RED2 is the one that matters for this change: it removes only the guard, keeps the committed_venv fix, and the new test fails alone. Host left on main with an empty git status.

#122183 reference stays as prose, not a closing keyword — five other PRs are chasing the same issue and this shouldn't close it out from under them.

@janviernine

Copy link
Copy Markdown

@Halldrix — correction accepted, pre-existing gap yours now closes rather than a regression it introduced. The RED2 isolation (guard-only revert, single failure) is exactly the evidence that matters. Thanks for carrying the sibling home.

@jonpol01 jonpol01 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The premise holds on main d0288be. With no generation committed, cron/scheduler_script.py::_windows_cron_python_invocation overlays selected_venv(repo) on the store Python. selected_venv falls back to base_venv, which is the in-tree venv. Switching to committed_venv is the right resolver and matches ethernet8023's c821ecd (pm/environments.py::committed_venv, activate_dependencies). The remaining problem is the arm with nothing committed. The comment says the child "keeps the interpreter it was handed", but the code still hands the child the bare store Python. That breaks scripts that work on main (finding 1). I checked with a probe that imports the real module under a temp HERMES_HOME (committed_venv really returns None, selected_venv really returns <repo>/.venv), stubs only resolve_store_python, and sets sys.platform = "win32" in the probe process only. Main at d0288be returns (<store>/python.exe, PYTHONPATH=<repo>:<repo>/.venv/...site-packages), which reproduces the bug. Head b86dffa returns (<store>/python.exe, PYTHONPATH=<repo>). The tests: scripts/run_tests.sh -j 2 tests/cron/test_cron_windows_venv_abi.py tests/cron/test_cron_script.py on head gives 31 passed, 5 skipped, and 4 of the skips are the new file on darwin. I read the new tests but could not run them on Windows here. Preflight: #123479 edits the same managed block (it moves these imports, renames the function to _cron_python_invocation and keeps selected_venv), so the two conflict textually, and merging #123479 afterwards would bring back the in-tree fallback. #123548 (the POSIX early-return in the same function) and #122742 (the _script_argv call site) touch neighbouring hunks.

  1. [blocking] The no-generation arm discards the handed interpreter and gives the child a bare store Python — cron/scheduler_script.py:164-167. Risk/repro: in the probe above, the handed interpreter is <tmp>/ownvenv/Scripts/python.exe (a uv venv with home=<tmp>/uvbase). Head returns interpreter=<tmp>/store/python.exe, PYTHONPATH=<repo>, and _script_argv then logs WARNING cron.scheduler: Windows cron script: venv site-packages Lib/site-packages not found; running without .pth processing on every spawn. The warning comes from _windows_cron_bootstrap_argv (head :277-279): the repo-only overlay has no site-packages entry, so it falls back to a relative Lib/site-packages. Why it matters: hermes_bootstrap.py:552-557 → pm/environments.py:280::_require_own_dependencies makes a bare store Python exit with hermes pm repair when nothing is committed. So any Hermes process running cron in this state runs on a non-store interpreter, which for a uv install is a venv carrying its own packages ("a venv/Nix interpreter keeps its own packages", c821ecd). The child gets none of them. A cron script has no hermes_bootstrap to refuse cleanly (compare the comment at pm/environments.py:359), so it runs and fails on its first third-party import. Pure-Python packages from the in-tree venv loaded on main (the PR body says so too); on head they fail. Suggested fix: return from the managed branch only when a generation is committed, and otherwise fall through to the existing uv overlay below, which keeps the handed venv's own base interpreter and its own site-packages. With that change the same probe returns interpreter=<tmp>/uvbase/python.exe, VIRTUAL_ENV=<tmp>/ownvenv, PYTHONPATH = <repo> + <tmp>/ownvenv/Lib/site-packages, and no warning. A handed bare store Python falls through to (python_exe, {}), a plain spawn. test_corrupt_record_degrades_instead_of_aborting_the_spawn would then assert on the handed interpreter instead of str(store).
  2. [nonblocking] The new test file runs on no CI lane — tests/cron/test_cron_windows_venv_abi.py:23-26 (module pytestmark). Risk/repro: python scripts/ci/list_os_marked_tests.py windows <tree containing only this file> prints No test files gate on 'windows' and exits 1. .github/workflows/tests-os.yml:121 builds the Windows lane's file list with exactly that script, and the Linux/macOS lanes skip the file. So the RED/GREEN evidence exists only on the author's host. tests/_fixtures/platform_gating.py and root AGENTS.md ("Use the marker, never a bare skipif") cover this case, and tests/cron/test_cron_script.py:166 already uses @pytest.mark.platforms("windows") for this function's Windows tests. Suggested fix: pytestmark = pytest.mark.platforms("windows"). Those tests could also move into tests/cron/test_cron_script.py next to the existing ones.

Minor

  • The comment at cron/scheduler_script.py:152-153 says site_packages raises when the committed tree has vanished. On Windows it is a pure path join (pm/environments.py:253). The vanished-tree check is the pyvenv.cfg probe inside _recorded_venv (pm/environments.py:192), which the try already covers, and dependency_site(...) sits outside the try. Please confirm the comment's intent and drop that clause.
  • Four tests exceed the repo's two-invariant-test bar. test_non_managed_install_keeps_its_own_venv_overlay passes on main by design (it is a control), and with finding 1 the no-generation and corrupt-record cases become one fall-through contract.
  • The body says every other selected_venv caller is guarded by runtime_facts_path(root).is_file(). pm/environments_adopt.py:29, hermes_cli/runtime_state.py:184 and pm/install.py:523 are not guarded that way. They only use the result for identity comparisons and never compose a child's PYTHONPATH from it, so the conclusion stands but the stated reason is wrong.

Related — #123479 (same block, see preflight). #122330 and #123149 make the same committed_venv switch in gateway/run.py::_ensure_windows_gateway_venv_imports. #123525 adds a POSIX sibling that also resolves through committed_venv.

Comment thread cron/scheduler_script.py Outdated
Comment on lines +164 to +167
pythonpath = [str(repo)]
if environment is not None:
pythonpath.append(str(dependency_site(environment)))
return str(managed_python), {"PYTHONPATH": os.pathsep.join(pythonpath)}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[blocking] The no-generation arm discards the handed interpreter and gives the child a bare store Python.

Risk/repro: the probe imports the real module under a temp HERMES_HOME, so committed_venv returns None. It stubs only resolve_store_python and sets sys.platform = "win32" in the probe process only. The handed interpreter is a uv venv shim, <tmp>/ownvenv/Scripts/python.exe. On this head the function returns interpreter=<tmp>/store/python.exe, PYTHONPATH=<repo>. _script_argv then logs venv site-packages Lib/site-packages not found; running without .pth processing on every spawn, because the repo-only overlay has no site-packages entry for _windows_cron_bootstrap_argv to find. When nothing is committed, _require_own_dependencies (pm/environments.py:280, called from hermes_bootstrap.py:554) refuses a bare store Python at boot. So the process running cron here is a non-store interpreter (for a uv install, a venv that carries its own packages). The child gets none of them, and pure-Python packages that loaded on main now fail.

Suggested fix: return from the managed branch only when a generation is committed, and otherwise fall through to the uv overlay below. The comment's "child keeps the interpreter it was handed" then becomes true: the same probe returns <tmp>/uvbase/python.exe with <tmp>/ownvenv/Lib/site-packages, and there is no warning.

Suggested change
pythonpath = [str(repo)]
if environment is not None:
pythonpath.append(str(dependency_site(environment)))
return str(managed_python), {"PYTHONPATH": os.pathsep.join(pythonpath)}
if environment is not None:
return str(managed_python), {
"PYTHONPATH": os.pathsep.join([str(repo), str(dependency_site(environment))])}

Comment on lines +23 to +26
pytestmark = pytest.mark.skipif(
sys.platform != "win32",
reason="_windows_cron_python_invocation is the Windows cron spawn path (#122183/#123650)",
)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nonblocking] This file runs on no CI lane. Risk/repro: python scripts/ci/list_os_marked_tests.py windows <tree containing only this file> prints No test files gate on 'windows' and exits 1. .github/workflows/tests-os.yml:121 builds the Windows lane's file list with that script, so the bare skipif is skipped on Linux/macOS and never imported on Windows. tests/cron/test_cron_script.py:166 gates this function's Windows tests with the repo's marker.

Suggested change
pytestmark = pytest.mark.skipif(
sys.platform != "win32",
reason="_windows_cron_python_invocation is the Windows cron spawn path (#122183/#123650)",
)
pytestmark = pytest.mark.platforms("windows")

…itted (NousResearch#122183)

The no-generation arm returned the bare store Python with a repo-only
overlay, which is worse than the main behavior it replaced: a store
Python with nothing committed is refused by _require_own_dependencies
with "hermes pm repair", and a cron script has no hermes_bootstrap to
refuse cleanly, so it runs and dies on its first third-party import. It
also stripped the overlay's site-packages entry, so
_windows_cron_bootstrap_argv warned on every spawn.

Fall through to the uv overlay instead: the handed venv keeps its own
base interpreter and its own packages, which is what worked on main.
The committed-generation arm is unchanged and still returns the store
Python with that generation's site-packages.

Also corrects the comment: site_packages is a pure path join on Windows
(pm/environments.py:250-255) and never raises there; the raise is the
pyvenv.cfg probe inside _recorded_venv:192, already covered here, and
dependency_site() is outside this try. (NousResearch#123668 review)
…#122183)

platforms("windows") instead of a bare skipif: list_os_marked_tests.py
gates the Windows lane on that marker, so the skipif version ran on no
CI lane at all -- the antipattern root AGENTS.md calls out.

Four tests collapse to the two invariants that actually matter: the
committed generation wins, and no-generation/unreadable-record both
fall through to the handed venv. The legacy non-managed test is gone --
it passes on main by design, and the fall-through test's positive
control now covers that path inside the managed arm.
@Halldrix

Copy link
Copy Markdown
Author

Fix 1 (blocking) — taken. The comment said the child "keeps the interpreter it was handed" and the code returned str(managed_python) — the bare store Python. Reproduced with your probe on W11 before touching anything:

interpreter : <tmp>\store\python.exe          <- not the handed venv
PYTHONPATH  : <repo>                          <- repo-only
bootstrap   : ['C:\job\script.py']            <- plain spawn
WARNING cron.scheduler: venv site-packages Lib\site-packages not found;
        running without .pth processing       <- on every spawn

You're right that this is worse than main, and the _require_own_dependencies chain is what makes it worse rather than merely different: a store Python with nothing committed is refused with hermes pm repair, and a cron script has no bootstrap to refuse it cleanly, so it runs and dies on the first third-party import. b3760eff7a returns from the managed branch only when a generation is committed; otherwise it falls through to the uv overlay. Your probe, verbatim, on the fixed head:

interpreter : <tmp>\uvbase\python.exe
PYTHONPATH  : <repo>;<tmp>\ownvenv\Lib\site-packages
VIRTUAL_ENV : <tmp>\ownvenv
bootstrap   : ['-c', "...site.addsitedir('<tmp>\ownvenv\Lib\site-packages')..."]

No warning, .pth processed. RED2 in the transcript below is exactly the isolation you asked for — only the fall-through reverted, committed_venv kept, one test fails.

Fix 2 (CI lane) — taken, and you were right that it was the marker. pytestmark = pytest.mark.platforms("windows") now; list_os_marked_tests.py windows . lists the file where it previously printed nothing. The skipif was the antipattern root AGENTS.md calls out by name.

Minor 1 — confirmed, comment corrected. site_packages is a pure join on Windows (pm/environments.py:250-255); the vanished-tree check is the pyvenv.cfg probe inside _recorded_venv:192, which the try does cover, and dependency_site() sits outside it. I had that clause wrong and it's gone.

Minor 3 — confirmed, and your reason is the better one. Restated: this function was the only selected_venv caller that composes a child's PYTHONPATH from the result, which is what turns the fallback into a live ABI mix. The others use it for identity comparison or guarded lookup, so the same fallback is inert there. My original runtime_facts_path().is_file() claim was false for environments_adopt.py:29, runtime_state.py:184 and install.py:523.

Minor 2 — taken, 4 tests → 2. Invariant 1 (committed generation wins), invariant 2 (no-generation and unreadable-record both land on the handed venv, checked as a table over both arms). The legacy non-managed test is gone — it passed on main by design, and the fall-through test's positive control now covers that path inside the managed arm.

Not addressed: #123479. You're right that it edits the same block and keeps selected_venv, so merging it after this would reintroduce the in-tree fallback. I've left it to the maintainers to sequence rather than reaching into someone else's PR.

W11 receipts on 30c649f16b: GREEN 2 passed; RED1 (prod at main) 2 failed; RED2 (only the fall-through) 1 failed, 1 passed. Host left on main, clean. Body re-synced to match.

NousResearch#122183)

Late adversarial gate on NousResearch#123668. Two findings, both mine:

1. BLOCKER. tools/environments/local_pythonpath.py:81 read selected_venv
   behind runtime_facts_path().is_file() with no try. That gate suppresses
   the FALLBACK, never the RAISE: _recorded_venv raises at pm/environments.py
   :180,185,189,193, and this runs inside _finalize_child_env -- the guard
   every spawn surface shares -- so a malformed facts.json crashed the same
   way the cron path did. The body claimed this caller "never hands it to a
   child"; it feeds the strip list of every child env. Same defect, same fix.

2. CONDITION. The bare except Exception downgraded payload_venv's
   "payload environment escapes its root" (pm/environments.py:92) -- a
   fail-closed security check -- to a warning. Narrowed to the repo's PM-read
   idiom (OSError, RuntimeError, ValueError; environments_adopt.py:30) and the
   escape check re-raised, on both sites.

Also writes the fall-through invariant the comment previously assumed: a
cron script gets sys.executable, and on a managed install with nothing
committed a booted process is by definition a venv interpreter
(_require_own_dependencies) carrying its own packages.
…uard (NousResearch#122183)

The fall-through had a second arm nobody asserted: a venv with no uv key
skips the overlay and keeps its own python.exe. That is the invariant that
keeps the arm from returning the bare store Python.

Two tests for local_pythonpath: a malformed record degrades to no runtime
venv, and payload_venv's escape check still propagates. The second test
caught the narrow-catch fix being applied to cron but not to its sibling.
@kshitijk4poor

Copy link
Copy Markdown

Thanks @Halldrix. Your three cron commits are cherry-picked into #123996 with authorship kept (merged with rebase).

One difference from your head: the try/except wrapper from 43751c7 is not carried. A corrupt or unreadable facts.json now raises, as in hermes_bootstrap and the gateway half of #123996, instead of being treated as "nothing committed". @janviernine's point stands that committed_venv can raise. The choice here is to fail closed in both places rather than fall back.

  • @jonpol01, blocking no-generation arm: covered. Your fall-through to the handed venv is in, and the probe shows the child keeps the handed interpreter and packages.
  • @jonpol01, CI lane: the test carries pytest.mark.platforms("windows") and is selected for the Windows lanes.
  • The stack is trimmed to one committed-generation invariant test per side.

Closing in favour of #123996. I'll post the merge SHA here when it lands.

kshitijk4poor pushed a commit that referenced this pull request Sep 26, 2026
…itted (#122183)

The no-generation arm returned the bare store Python with a repo-only
overlay, which is worse than the main behavior it replaced: a store
Python with nothing committed is refused by _require_own_dependencies
with "hermes pm repair", and a cron script has no hermes_bootstrap to
refuse cleanly, so it runs and dies on its first third-party import. It
also stripped the overlay's site-packages entry, so
_windows_cron_bootstrap_argv warned on every spawn.

Fall through to the uv overlay instead: the handed venv keeps its own
base interpreter and its own packages, which is what worked on main.
The committed-generation arm is unchanged and still returns the store
Python with that generation's site-packages.

Also corrects the comment: site_packages is a pure path join on Windows
(pm/environments.py:250-255) and never raises there; the raise is the
pyvenv.cfg probe inside _recorded_venv:192, already covered here, and
dependency_site() is outside this try. (#123668 review)

(cherry picked from commit b3760ef)
@kshitijk4poor

Copy link
Copy Markdown

Merged: #123996 landed on main as 86f449b.

@alt-glitch alt-glitch added os/windows Native Windows behaviour or breakage: PowerShell install, Windows paths, cmd.exe, WSL-vs-native diff and removed platform/windows labels Oct 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/install-update Installer, updater, packaging, wheels, doctor comp/cron Cron scheduler and job management os/windows Native Windows behaviour or breakage: PowerShell install, Windows paths, cmd.exe, WSL-vs-native diff P1 High — major feature broken, no workaround sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-platform-windows Sweeper risk: may break or behave differently on native Windows type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants