Repository navigation
fix(macos): re-land dylib-complete TCC interpreter anchor (salvage #95605) - #96647
Merged
kshitijk4poor merged 6 commits intoAug 28, 2026
Merged
kshitijk4poor merged 6 commits into
kshitijk4poor merged 6 commits into
Conversation
The first landing (NousResearch#95131/NousResearch#95478, reverted in NousResearch#95563) copied the uv-store interpreter into venv/bin/python so TCC grants would stick to a stable path. On real Macs that copy bricked every hermes command two ways: dynamically-linked builds died in dyld because @executable_path/../lib/libpython resolved into venv/lib/ (NousResearch#95425), and alias symlinks to the copy made CPython getpath lose the venv prefix (NousResearch#95541, ModuleNotFoundError: encodings). Re-land: - Keep the signed real-file copy of bin/python (identifier-pinned via _macos_sign_managed_python). - Materialize python3 / python3.N as real-file copies, never symlinks. Copies boot on every build we could reproduce and keep the TCC identity. - Hardlink store libpython* into venv/lib/ when present (copy across devices). Existing LC_RPATH already points there. - Pre-install boot gate: launch the staged copy, demand encodings plus the venv prefix, abort and leave the live venv untouched on failure. Doctor reports/installs the new anchor (the revert-era heal is removed). Update refreshes it after a successful code swap. Tests cover layout, idempotence, predecessor-symlink repair, libpython hardlink, boot-gate refusal, and a macos_only real-interpreter E2E. Closes NousResearch#95596.
The gate is the never-brick guarantee, so each direction gets its own test: nonzero exit (dyld/encodings crash), build-time prefix leak, timeout, and the deliberate OSError skip (a binary that cannot execute here means the symlinked venv was equally dead — installing cannot make things worse).
Review fixes from kokhlo's live-hardware review: - The boot-gate probe now runs with PYTHONHOME / PYTHONPATH / PYTHONSTARTUP / __PYVENV_LAUNCHER__ scrubbed: an inherited PYTHONHOME=<venv> boots a staged copy that would otherwise die with "No module named 'encodings'", papering over the exact prefix failure the gate exists to catch. - OSError is split by errno: ENOENT/ENOEXEC (fixtures, foreign-arch images) still skip; EACCES after our own chmod now refuses the install instead of silently accepting a broken copy. - Marker writes and both marker comparisons go through os.path.realpath, so the managed-runtime layout (cpython-3.11-macos-* symlinked to cpython-3.11.15-macos-*) no longer reports stale on a fresh install. Tests: +3 (env-scrub spy, EACCES refusal, symlinked-home state). 100 passed in the module + doctor neighborhoods.
…er-last Follow-up to the NousResearch#95605 salvage, closing the review findings: - _copy_alias no longer swallows OSError silently: it warns (a leftover alias symlink is the exact NousResearch#95541 crash shape) and reports failure. - Alias staging uses mkstemp (unique names) so concurrent ensures (update + doctor --fix) can never promote a truncated interim copy. - The anchor marker is written LAST and atomically (write-then-rename): it now asserts the whole layout (anchor + aliases) is complete, so a partially-materialized alias set can never read 'active' in doctor — the next ensure retries the install instead. - /.hermes-runtime/python/ store marker is derived from managed_uv._RUNTIME_DIR_NAME instead of a hardcoded string. 5 new regression tests.
kshitijk4poor
force-pushed
the
salvage/95605-tcc-anchor-reland
branch
from
August 27, 2026 21:09
f3e20e2 to
e447609
Compare
|
Verified the hardening commit (e447609) — all four changes are sound. Making alias failures visible is right (a leftover symlink is exactly the #95541 crash shape); the atomic marker-last write closes the torn-read/partial-layout window; mkstemp staging removes the concurrent-ensure collision; deriving the store marker from |
/simplify-code reuse finding: _write_marker reimplemented the mkstemp->write->os.replace pattern that utils.atomic_write_text already provides as the repo's shared atomic-text-write helper (and the shared version adds fsync + cross-device/busy-file fallbacks).
kshitijk4poor
enabled auto-merge (rebase)
August 28, 2026 03:30
kshitijk4poor
added a commit
that referenced
this pull request
Aug 28, 2026
Open
20 tasks
melon-xf
added a commit
to melon-xf/hermes-agent
that referenced
this pull request
Sep 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Salvage of #95605 (@zengzheqing) onto current main with a hardening follow-up: the dylib-complete re-land of the macOS TCC interpreter anchor, live-verified on real macOS hardware (Apple Silicon), plus fixes for three review findings around alias-failure observability and concurrency.
The contributor's three commits are cherry-picked with authorship preserved and apply cleanly (zero conflicts — no churn on these files since the PR branched).
What the anchor does (from #95605)
Re-land of the mechanism reverted in #95563, against that revert's criteria (dylib-complete design, live-verified on macOS hardware):
venv/bin/pythonis a real-file copy of the uv-store interpreter, identifier-DR signed (com.nousresearch.hermes.managed-python) — TCC grants survive Python patch bumps ([Bug]: macOS permission prompts return after every update — TCC grants keyed to the uv-managed Python interpreter path, not the app bundle #85345).python3/python3.Nare real-file copies, never symlinks — symlink aliases lose the venv prefix on python-build-standalone builds ([Bug]: macOS TCC anchor re-points venv python3 aliases to symlinks that cannot find the stdlib — CLI dies with ModuleNotFoundError: encodings #95541).libpython*is hardlinked intovenv/lib/so@executable_path/../libresolves — no more dyld death ([Bug]: macOS TCC anchor copies the uv interpreter into venv/bin without libpython — every hermes command dies in dyld #95425).import encodings+sys.prefix == <venv>, scrubbed env) beforeos.replace. A bad anchor can never brick update/doctor again.hermes doctorreports/installs the anchor;hermes updaterefreshes it. The revert-era heal (check_macos_tcc_anchor_removed) is removed — first-generation broken copies are repaired in place.Follow-up hardening (our commit, review findings on the salvage)
_copy_aliasno longer swallowsOSErrorsilently — it warns and reports failure (a leftover alias symlink is the exact [Bug]: macOS TCC anchor re-points venv python3 aliases to symlinks that cannot find the stdlib — CLI dies with ModuleNotFoundError: encodings #95541 crash shape; previously invisible).mkstempunique names so concurrent ensures (update +doctor --fix) can never promote a truncated interim copy./.hermes-runtime/python/store marker is derived frommanaged_uv._RUNTIME_DIR_NAMEinstead of a drift-prone hardcoded string.Validation (live on macOS arm64)
pytest test_macos_tcc_anchor.py test_doctor.pymacos_onlyE2E)python/python3/python3.12boot, venv prefix, libpython hardlinked (same inode)Library not loaded @executable_path/../lib/libpython)ensure_tcc_anchor; all entry points boot afterbin/pythonbyte-identicalruff check+git diff --checkField confirmation of the alias-copy approach: #95551 comment from @fred0m (production install, #95541 signature, recovered with the same mechanism). Reviewer findings from @kokhlo on #95605 (gate env scrub, EACCES refusal, marker realpath) were already addressed by the contributor with regression tests.
Credit
Anchor design and implementation by @zengzheqing (#95605) — commits cherry-picked with authorship preserved. Builds on @kokhlo's review (#95754 closed as duplicate) and the failure-mode mapping in #95358 / #95476 / #95551.
Closes #95596. Closes #95605. Refs #85345, #95425, #95541, #95563.
Post-review fold (/simplify-code on the final stack)
refactor(macos):_write_markernow uses the sharedutils.atomic_write_text(repo's single atomic-text-write implementation, adds fsync + cross-device fallbacks) instead of a bespoke mkstemp/replace block.macos_tcc_anchor.pyto the contributor base fails the 3 new-behavior guards and passes the sync-contract test (correct split); full suite 39/39 green after restore._RUNTIME_DIR_NAMEprivate import is the right coupling (no public bare-name accessor; sync test binds it); +13ms lazy-import cost onhermes doctoronly — not material.