fix(lazy_deps): range huggingface-hub + never downgrade a shared dependency - #70115
Closed
teknium1 wants to merge 2 commits into
Closed
fix(lazy_deps): range huggingface-hub + never downgrade a shared dependency#70115teknium1 wants to merge 2 commits into
teknium1 wants to merge 2 commits into
Conversation
…aking Hindsight tool.trace_upload pinned huggingface-hub==1.2.3, but huggingface-hub is a shared dependency: transformers (via sentence-transformers, the Hindsight local-embeddings provider) requires huggingface-hub>=1.5.0,<2.0. active_features() flags a feature as active from mere package presence, so having sentence-transformers installed marks tool.trace_upload active even for users who never ran a trace upload. On the next hermes update, _refresh_active_lazy_features() sees the ==1.2.3 pin unsatisfied and downgrades the shared package, breaking Hindsight startup with ImportError: huggingface-hub>=1.5.0,<2.0 is required. Widen the pin to huggingface-hub>=1.2.3,<2.0 (ranges are the norm in LAZY_DEPS; the == pin was the outlier): every transformers-compatible version now satisfies the spec, so the refresh treats it as current instead of downgrading, and a fresh lazy install resolves to a current 1.x. The HfApi surface trace upload uses (whoami / create_repo / upload_file) is stable across the whole 1.x line. Tests pin the invariant: the trace_upload spec must admit every version transformers accepts (loud failure if someone re-pins it into conflict), and feature_missing() must report a newer in-range hub as satisfied. Fixes #60783
…r huggingface-hub An exact ==1.2.3 pin on huggingface-hub (feature tool.trace_upload) force-downgraded the shared venv on every lazy refresh whenever the core embedding stack (transformers/sentence-transformers, used by local/local_embedded Hindsight) had installed a newer version — transformers 5.x requires huggingface-hub>=1.5,<2.0, so the downgrade made sentence_transformers unimportable and the embedded Hindsight daemon abort at startup (silent memory loss until noticed). Two layers: - Track the compatibility range the trace-upload client actually needs (>=1.5,<2.0) instead of an exact pin, so an already-healthy shared version satisfies the spec and is left alone. - Add a general no-downgrade guard in _is_satisfied: a lazy, opt-in backend must never move an already-installed package backwards; treat 'installed newer than the pin allows' as satisfied and warn to widen the pin. Legitimate upgrades (installed below the spec) are unaffected. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MN8RMDLwxCfFxwtADoEJJf
Contributor
૮ >ﻌ< ა ci reviewrunning on 3ab819d Still running 1 job: ℹ️ InfoDesktop E2E visual evidence · View test artifacts · View job1 visual diff. inline evidence upload failed. Failed to upload diff-665a0833239e-onboarding-overlay-diff.png with gh image (exit code 1): Error uploading /home/runner/work/_temp/e2e-evidence/diff-665a0833239e-onboarding-overlay-diff.png: step 0 (get upload token): uploadToken not found on repo page — do you have write access to NousResearch/hermes-agent? (or, if NousResearch enforces SAML SSO, authorize at https://github.com/orgs/NousResearch/sso) |
alt-glitch
added
type/bug
Collaborator
12 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
hermes updatecan no longer downgradehuggingface-hub(or any shared dependency) out from under its other consumers — fixing the Hindsight local-embeddings breakage in #60783.Root cause:
LAZY_DEPS["tool.trace_upload"]exact-pinnedhuggingface-hub==1.2.3.active_features()marks a feature active from mere package presence (hub is a transitive of faster-whisper/tokenizers in the core lock), so every post-update lazy refresh reinstalled the stale pin, downgrading hub below the>=1.5.0,<2range that transformers/sentence-transformers (Hindsight local embeddings) require. The daemon then failed on startup with a misleading "sentence-transformers is required" error.Changes
tools/lazy_deps.py:tool.trace_uploadpin widened tohuggingface-hub>=1.5.0,<2.0(the range transformers requires; HfApi surface used by trace upload —whoami/create_repo/upload_file— is stable across 1.x)tools/lazy_deps.py: generic no-downgrade guard in_is_satisfied()— any lazy spec whose only path to satisfaction is moving an already-installed package backwards is treated as satisfied, the installed version is left in place, and a maintainer warning is logged. Fixes the class, not just this pin.tests/tools/test_lazy_deps.py: invariant test that the trace-upload spec admits transformers' whole accepted range (fails loudly if ever re-pinned into conflict); regression test that a newer compatible hub is treated as currentValidation
hermes updatewith hub 1.24.0 installed==1.2.3spec vs installed 1.24.0scripts/run_tests.sh tests/tools/test_lazy_deps.py— 66/66 greenfeature_missing()empty, no-downgrade guard exercised against the old pinCredit
Salvages #60797 by @falkoro (earliest submitter — range fix + invariant tests; floor raised from 1.2.3 to transformers' 1.5.0 during conflict resolution) and #68008 by @spiky02plateau (no-downgrade guard). Both authorships preserved via cherry-pick. Same-cluster PRs #66261 (@stigrunar) and #63936 (@lipton-agent-hermes) reached the same range independently.
Fixes #60783.
Infographic