fix(doctor): use the bounded wrapper scanner for orphan profile aliases - #84464
fix(doctor): use the bounded wrapper scanner for orphan profile aliases#84464AlexxRussell wants to merge 2 commits into
Conversation
Doctor read every entry in the profile wrapper directory in full via Path.read_text() while looking for orphan profile aliases. That directory is normally ~/.local/bin, which also holds large unrelated binaries, so on a small host the scan retained hundreds of MiB and Doctor was OOM killed. Reuse hermes_cli.profiles.build_alias_map(), the scanner already used for profile listing. It reads at most _WRAPPER_READ_LIMIT bytes per candidate and skips binaries, then orphans are the mapped profiles that no longer exist.
fix(doctor): use the bounded wrapper scanner for orphan profile aliases Reviewed A few observations:
|
build_alias_map() is keyed by profile, so it holds exactly one alias per profile. Reading orphan wrappers back out of it collapsed several stranded wrappers that name the same missing profile into a single warning. That is not a rare shape: adding a custom alias leaves both the profile-named wrapper and the alias wrapper on disk, so removing the profile strands two files and Doctor named only one of them. Extract the bounded per-wrapper scan into iter_wrapper_aliases() and build build_alias_map() on top of it. The alias map keeps its exact previous semantics and its two callers are untouched, while Doctor can now walk every wrapper. The 8 KB read cap and the binary skip that this branch introduced are unchanged, so the unbounded read stays gone. Tests: two stranded wrappers now produce two warnings (fails without this change, reporting only one). Also pins the case-folding behaviour raised in review: a mixed-case target is reported under the canonical lowercase id, and case never decides whether a profile counts as missing, since profile_exists() already normalises its argument.
|
Point 2 is correct and is now fixed in f782ae1.
Rather than iterate the wrapper dir separately, which would stand up a second scanner beside the first, I extracted the bounded per-wrapper scan into On point 1, the normalization is not narrowing which profiles get reported. # hermes_cli/profiles.py:382
def profile_exists(name: str) -> bool:
canon = normalize_profile_name(name)The old path called I left out the old versus new parity test, since it would pin the scanner this PR exists to remove, and the remaining differences are the deliberate ones listed in the description. On the largest of those: alias names are validated against |
What does this PR do?
Doctor's orphan profile alias check read every entry in the profile wrapper directory in full:
That directory is normally
~/.local/bin, which on a real machine also holds large unrelated binaries (uv, Python, node, ffmpeg).Path.read_text()pulls each of those entirely into memory. On a 1 GB VPS running Hermes this is fatal: Doctor was OOM killed with SIGKILL while running that loop.hermes_cli/profiles.pyalready ships a scanner built for exactly this directory.build_alias_map()reads at most_WRAPPER_READ_LIMIT(8192) bytes per candidate, opens witherrors="strict"so binaries raiseUnicodeDecodeErrorand are skipped, and is already the path used for profile listing. Doctor should reuse it rather than keep a second, unbounded scan of the same directory.Orphans then fall out of the existing map: the profiles it resolved that no longer exist.
Related Issue
No existing issue. Found in production on a memory constrained host.
Type of Change
Changes Made
hermes_cli/doctor.py: added_find_orphan_profile_aliases(), which readsbuild_alias_map()and keeps the entries whose profile is missing.hermes_cli/doctor.py:run_doctor()now iterates that helper instead of walking the wrapper directory itself. Dropped the now unusedprofile_existsandreimports from that block.tests/hermes_cli/test_doctor_orphan_alias_scan.py: new file, 5 tests.Relationship to #77058
#77058 also touches this block, and the two do not overlap. That PR replaces the regex with
_profile_from_wrapper()to support quoted aliases, but keepswrapper.read_text(encoding="utf-8"), so the unbounded read remains (andshlex.split()then runs over the whole file). This PR is the memory fix and does not change parsing.If #77058 lands first I am happy to rebase. My suggestion in that case is that quoted alias support belongs inside
build_alias_map()rather than in a second parser in Doctor, so both Doctor and profile listing get it from one place. Happy to do that follow up if a maintainer prefers it.How to Test
hermes -p <profile>marker sits past the first 8192 bytes.hermes doctor.Behavioural differences I am aware of
Disclosing these rather than leaving them to be found in review. All three follow from reusing the shared scanner:
build_alias_map()skips entries with a suffix on POSIX and requires.baton Windows. That matches how Hermes creates wrappers, but a hand renamed wrapper such asmyalias.shwould no longer be reported as an orphan. The old loop checked every file.build_alias_map()keeps a single alias per profile (a custom alias wins over the profile named one). If two wrappers point at the same missing profile, the old code printed two warnings and this prints one.foo) rather than the file name (foo.bat).If any of these matter, the alternative is a new bounded helper in
profiles.pythat returns every matching wrapper rather than a deduplicated map. I went with reuse because a second scanner of the same directory is what caused this bug.Verification
Run against current main with
pytest:hermes_cli/doctor.pygives 2 failed, 3 passed. The two failures are behavioural, not import errors:test_profile_marker_after_read_limit_is_ignored(a marker past the read limit is visible again)test_wrapper_dir_entries_are_never_read_whole(Path.read_text()is called on a wrapper dir entry again)The other three pass either way by design, since they assert that existing behaviour is preserved.
tests/hermes_cli/test_profiles.pyandtests/hermes_cli/test_doctor.py: 97 passed, 2 skipped, 0 failed.All five tests drive the real
run_doctorcode path and assert on its output rather than calling the new helper directly.Checklist
Code
I've read the Contributing Guide
My commit messages follow Conventional Commits
I searched for existing PRs to make sure this isn't a duplicate (see the fix(doctor): support local Mem0 and quoted profile aliases #77058 section above)
My PR contains only changes related to this fix
I've run
pytest tests/ -qand all tests passNot ticked deliberately. I ran the new file plus the two neighbouring suites listed above. I did not run the whole tree, so I am not claiming it. CI is the authority here.
I've added tests for my changes
I've tested on my platform: Debian 12 (the affected host) and macOS 15
Documentation & Housekeeping
cli-config.yaml.exampleif I added/changed config keys, or N/A. N/A, no new keys.CONTRIBUTING.mdorAGENTS.md, or N/A. N/A..batand stem handling both come from the shared scanner.