Skip to content

feat(update): prefer locked dependency sync - #5984

Open
counterposition wants to merge 1 commit into
NousResearch:mainfrom
counterposition:chore/banish-uv-pip-subcommand
Open

feat(update): prefer locked dependency sync#5984
counterposition wants to merge 1 commit into
NousResearch:mainfrom
counterposition:chore/banish-uv-pip-subcommand

Conversation

@counterposition

@counterposition counterposition commented Apr 8, 2026

Copy link
Copy Markdown
Contributor

Rationale

Hermes-owned installers and CI already use uv sync --locked, but source-update dependency refreshes still ran uv pip install -e ".[all]". That re-resolved transitive dependencies from package indexes on every update instead of using the reviewed, hash-bearing lockfile.

This PR narrows the remaining gap without changing Termux dependency policy or the installer environment layout.

Summary

  • Makes uv sync --locked --inexact --extra all the primary dependency tier for uv-capable, non-Termux source-update flows:
    • normal hermes update;
    • ZIP update;
    • interrupted-install recovery; and
    • unhealthy-venv repair.
  • Falls back to the existing editable uv pip install cascade when uv.lock is absent, stale, or cannot be synced.
  • Keeps --inexact so lazy-installed backends, memory-provider SDKs, and user-added packages are not pruned.
  • Keeps every update tier on the installer-managed venv layout protected by the existing Windows concurrent-process and executable-quarantine guards.
  • Normalizes UV_PROJECT_ENVIRONMENT, VIRTUAL_ENV, and UV_PYTHON for both sync and fallback paths so inherited interpreter settings cannot redirect an update or repair.
  • Pins UV_PYTHON to the managed environment interpreter when it exists and removes a hostile inherited value while the environment is being created.
  • Verifies declared core dependencies and console-script shims after a successful locked sync.
  • Updates doctor/dashboard repair guidance and the update documentation to describe the locked-sync-first behavior.

Termux scope

Termux deliberately does not enter the new uv sync tier. It remains on the existing uv pip install -e ".[termux-all]" path because the Android-compatible psutil prebuild and the broader native-sdist policy are not lockfile-sync safe yet.

Real-device Termux findings from this comment are tracked separately:

The PR includes regression coverage for both the helper-level Termux sync exclusion and the interrupted-recovery path's termux-all selection.

Test plan

  • Rebased onto current upstream/main as a single commit
  • bash -n setup-hermes.sh
  • bash -n scripts/install.sh
  • Python byte-compilation for all changed Python files
  • Ruff on all changed Python files
  • Focused affected suite: 180 passed
  • Broader update/supporting suite excluding the known macOS systemd baseline failures: 246 passed
  • The four tests/hermes_cli/test_gateway_service.py macOS failures reproduce unchanged on a clean upstream/main worktree (185 passed, 4 failed there and on this branch)
  • git diff --check

Not run locally: the complete repository test suite. GitHub CI will exercise the full configured matrix.

@blasai1739217-cmyk

Copy link
Copy Markdown

Friendly bump on this PR in case it fell through the cracks \u2014 would love a review when someone has a minute. Thanks!

@counterposition

Copy link
Copy Markdown
Contributor Author

Friendly bump on this PR in case it fell through the cracks \u2014 would love a review when someone has a minute. Thanks!

Dumb clanker. This PR was opened less than 2 hours ago. There's no way to do a "friendly" bump this early.

Maintainers, please take your time. It's a large PR that touches numerous scripts as well as CI workflows.

@counterposition
counterposition force-pushed the chore/banish-uv-pip-subcommand branch from afb6806 to 74be48e Compare April 8, 2026 15:39
@counterposition
counterposition force-pushed the chore/banish-uv-pip-subcommand branch 3 times, most recently from 0ac6344 to 33c8405 Compare April 11, 2026 19:11
@alt-glitch alt-glitch added type/refactor Code restructuring, no behavior change P3 Low — cosmetic, nice to have area/config Config system, migrations, profiles labels Apr 30, 2026
@counterposition
counterposition force-pushed the chore/banish-uv-pip-subcommand branch 2 times, most recently from 2248e7f to 070563b Compare May 9, 2026 01:49

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the broad project-flow cleanup. The source-update premise is still live on current main: normal updates use uv pip at hermes_cli/main.py:9909-9921, and ZIP updates do so at hermes_cli/main.py:6353-6357.

Problems

  • The proposed .venv migration must be reconciled with newer Windows update protections. Current main's protected scripts lookup is hard-coded to PROJECT_ROOT / "venv" (hermes_cli/main.py:7214-7220) and is used by the update guard (hermes_cli/main.py:9453-9459). Switching installer output to .venv without porting those paths would bypass the lock/concurrent-instance protection.
  • tests/test_project_metadata.py:51 rejects any mention of --all-extras in selected docs, rather than validating actual install commands. That will reject legitimate explanatory text.

Suggested changes

  • Preserve or comprehensively generalize the current venv safety and recovery paths to the chosen environment directory, with end-to-end coverage.
  • Narrow or remove the literal documentation scan.

This is an automated hermes-sweeper review.

Comment thread scripts/install.ps1
Comment thread tests/test_project_metadata.py
@teknium1 teknium1 added sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-automation Sweeper risk: may affect CI, automerge, label sync, or maintainer automation sweeper:risk-platform-windows Sweeper risk: may break or behave differently on native Windows sweeper:blast-broad Sweeper blast radius: broad — a core path most sessions hit labels Jul 12, 2026
@counterposition
counterposition force-pushed the chore/banish-uv-pip-subcommand branch from 070563b to 2b036ee Compare July 13, 2026 21:43
@counterposition

Copy link
Copy Markdown
Contributor Author

Rescoped this PR per the review.

Most of the original 30-file migration has since landed on main independently — the installers, CI, and the Dockerfile are already uv sync --locked first. I've also dropped the .venv standardization (agreed it needs reconciling with the venv-based Windows update protections, so the environment directory stays venv) and the literal --all-extras doc-scan test.

What remains is the one still-live gap: hermes update (normal, ZIP, and recovery paths) still refreshed dependencies with an unlocked uv pip install -e .[all], re-resolving from PyPI with none of the lockfile's hash verification. This PR now makes uv sync --locked --inexact --extra all the primary tier for the update flow, pointed at the existing venv via UV_PROJECT_ENVIRONMENT, with the Windows exe-quarantine preserved. --inexact matters: a plain sync would prune lazily-installed backends and anything users added by hand. Termux and lockfile-less checkouts skip the sync tier, and any sync failure falls back to the existing pip cascade unchanged. Doctor and dashboard repair hints now point at the same locked-sync repair.

Copy link
Copy Markdown
Contributor

Real-device Termux evidence relevant to this uv migration: on native Android/aarch64 with Python 3.11.15 and uv 0.11.29, the current patched-psutil helper compiles successfully but emits psutil-7.2.2-cp36-abi3-linux_aarch64.whl, which uv rejects for the Android interpreter. A global _PYTHON_HOST_PLATFORM=android_... is not viable because it also changes uv's interpreter probe and fails with Unknown operating system: android_.... I opened focused draft #67136, which applies the Android tag package-locally inside the extracted psutil sdist and includes behavior-level wheel-filename coverage.

The same migration run also exposed additional native-sdist considerations that this broad uv conversion should account for before treating Termux as covered:

  • MarkupSafe, httptools, PyYAML, cffi, Pillow, and ruamel-yaml-clib can compile locally but initially emit linux_aarch64 wheels that uv rejects for Android unless the build is made pure-Python or receives a package-scoped Android platform tag.
  • uvicorn[standard] pulls httptools/watchfiles; the Termux extra's Telegram webhook path pulls tornado. Plain uvicorn avoided the standard-extra native dependencies in the successful recovery, but that policy decision belongs in this migration rather than fix(termux): emit Android-tagged psutil wheels #67136.
  • pydantic-core, jiter, cryptography, and watchfiles can fall back to Rust source builds. Serial builds with an isolated CARGO_HOME avoided repeated Cargo registry .cargo-ok: File exists extraction failures on the device.
  • Adding the TUR Python index globally caused its available cryptography versions to shadow the pinned PyPI version under uv's default first-index policy. A production installer should use explicit package/index routing rather than unsafe-best-match.

I kept #67136 to two files and only the psutil helper because the fast-version NameError already has #62096, Python 3.14 is tracked separately in #59877/#48723, and the remaining uv dependency-policy work overlaps this PR directly.

@alt-glitch alt-glitch added comp/cli CLI entry point, hermes_cli/, setup wizard and removed sweeper:risk-automation Sweeper risk: may affect CI, automerge, label sync, or maintainer automation sweeper:risk-platform-windows Sweeper risk: may break or behave differently on native Windows labels Jul 18, 2026
@counterposition
counterposition force-pushed the chore/banish-uv-pip-subcommand branch from 2b036ee to e9b3e6d Compare July 18, 2026 21:39
@counterposition counterposition changed the title chore: migrate Hermes project flows from uv pip to uv sync feat(update): prefer locked dependency sync Jul 18, 2026

Copy link
Copy Markdown
Contributor Author

Thanks — this real-device evidence was useful, and it exposed that the PR title/body were still describing the old broad migration.

I've rebased and refreshed the PR. The current patch does not move Termux to uv sync: the new locked-sync tier explicitly gates out Termux, and both normal recovery and the helper-level regression keep Android on uv pip install -e ".[termux-all]". I also tightened the shared uv environment handling so inherited UV_PYTHON cannot redirect either the sync or fallback tier, while keeping all update paths on the Windows-protected venv layout.

The focused psutil wheel-tag fix remains in #67136. I opened #67150 for the remaining Android dependency-policy work you identified: plain vs standard uvicorn, native-sdist wheel tags, serialized/isolated Cargo builds, and package-specific TUR/PyPI routing. That keeps those real Termux gaps visible without folding them into this non-Termux update change.

@counterposition
counterposition force-pushed the chore/banish-uv-pip-subcommand branch from e9b3e6d to eff9f28 Compare July 20, 2026 19:47
@alt-glitch alt-glitch added type/feature New feature or request area/install-update Installer, updater, packaging, wheels, doctor and removed type/refactor Code restructuring, no behavior change area/config Config system, migrations, profiles labels Jul 20, 2026
Source updates still refreshed Hermes with uv pip, bypassing the locked
dependency set already used by installers and CI.

Prefer uv sync --locked --inexact for non-Termux update, recovery,
ZIP, and repair flows. Fall back to the existing editable-install cascade
when the lockfile is missing or the sync cannot run.

Keep every tier on the managed venv layout protected by the Windows
update guards. Sanitize UV_PROJECT_ENVIRONMENT, VIRTUAL_ENV, and
UV_PYTHON for both sync and fallback so inherited interpreter settings
cannot redirect a repair.

Termux deliberately stays on its curated termux-all uv pip path. Update
the repair guidance and cover sync selection, fallback, environment
normalization, console-script verification, and Termux recovery.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@counterposition
counterposition force-pushed the chore/banish-uv-pip-subcommand branch from eff9f28 to 502e53b Compare July 25, 2026 02:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/install-update Installer, updater, packaging, wheels, doctor comp/cli CLI entry point, hermes_cli/, setup wizard P3 Low — cosmetic, nice to have sweeper:blast-broad Sweeper blast radius: broad — a core path most sessions hit sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants