test(skills): cover get_external_skills_dirs input validation branches - #22273
Closed
wesleysimplicio wants to merge 2 commits into
Closed
test(skills): cover get_external_skills_dirs input validation branches#22273wesleysimplicio wants to merge 2 commits into
wesleysimplicio wants to merge 2 commits into
Conversation
Add 11 pytest cases for the input-validation and entry-normalization
branches in agent.skill_utils.get_external_skills_dirs that the existing
mtime-cache happy-path suite leaves uncovered:
- YAML root not a mapping
- skills section not a mapping
- external_dirs as single string (coerced)
- external_dirs as nested dict (rejected)
- ${VAR} env-var expansion in entries
- ~ expansion in entries
- relative entries resolved against HERMES_HOME
- local ~/.hermes/skills/ silently dropped
- duplicates collapsed
- non-existent paths skipped
- empty / whitespace-only entries skipped
Uses an isolated_hermes_home fixture (tmp_path + monkeypatch of
HERMES_HOME / HOME / Path.home) so cases never touch the real
~/.hermes/. Cache is cleared before and after each case to keep
state hermetic.
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a focused pytest suite to cover the defensive/input-validation and path-normalization branches of agent.skill_utils.get_external_skills_dirs, complementing the existing cache/mtime-oriented tests.
Changes:
- Introduces 11 new tests covering malformed
config.yamlshapes (non-mapping root, non-mappingskills, invalidexternal_dirstypes). - Adds coverage for normalization behavior (string coercion to list, env/tilde expansion, relative path resolution under
HERMES_HOME, deduping, skipping empty/nonexistent/local skills dir entries).
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
|
||
| def test_tilde_in_entry_is_expanded(isolated_hermes_home): | ||
| home, root = isolated_hermes_home | ||
| # Path.home() is patched to tmp_path by the fixture. |
get_external_skills_dirs() uses os.path.expanduser(), which on POSIX consults the HOME env var — not Path.home() (which the fixture does not patch). Update the comment to reflect what the fixture actually sets up.
Contributor
Author
|
Closing non-fix PR as requested — mantendo apenas PRs de fix. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Adds 11 pytest cases pinning the input-validation and entry-normalization branches of
agent.skill_utils.get_external_skills_dirsthat the existing mtime-cache happy-path suite (test_external_skills_dirs_cache.py) leaves uncovered.Root cause
The detailed rationale from the original PR body is preserved below. This template update keeps the review structure consistent with #29640.
Fix
Why this shape
get_external_skills_dirsis on the cold-start hot path (called once per skill during banner / tool-registry scans) and silently swallows malformedconfig.yamlshapes — a YAML root that isn't a mapping, askills:value that isn't a mapping, anexternal_dirsthat isn't a list, etc. Today none of those defensive branches have a regression test, so a future cleanup that drops the wrongisinstancecheck would pass CI green and break user configs at runtime.Tests
Original body
Related PRs / issues
Original body
Summary
Adds 11 pytest cases pinning the input-validation and entry-normalization branches of
agent.skill_utils.get_external_skills_dirsthat the existing mtime-cache happy-path suite (test_external_skills_dirs_cache.py) leaves uncovered.What Changed
Fluxo
A mudança continua seguindo o fluxo original descrito na seção preservada abaixo, sem ampliar o escopo funcional deste PR.
Visão
A padronização melhora a revisão, reduz ruído e evita deriva de formatação entre PRs abertos.
Test Plan
Original body
What
Adds 11 pytest cases pinning the input-validation and entry-normalization branches of
agent.skill_utils.get_external_skills_dirsthat the existing mtime-cache happy-path suite (test_external_skills_dirs_cache.py) leaves uncovered.Why
get_external_skills_dirsis on the cold-start hot path (called once per skill during banner / tool-registry scans) and silently swallows malformedconfig.yamlshapes — a YAML root that isn't a mapping, askills:value that isn't a mapping, anexternal_dirsthat isn't a list, etc. Today none of those defensive branches have a regression test, so a future cleanup that drops the wrongisinstancecheck would pass CI green and break user configs at runtime.Coverage added
skill_utils.pynot isinstance(parsed, dict)test_yaml_root_not_mapping_returns_emptynot isinstance(skills_cfg, dict)test_skills_section_not_mapping_returns_emptyisinstance(raw_dirs, str)coerciontest_external_dirs_as_single_string_is_coerced_to_listnot isinstance(raw_dirs, list)test_external_dirs_as_invalid_type_returns_emptyos.path.expandvarstest_environment_variable_in_entry_is_expandedos.path.expandusertest_tilde_in_entry_is_expandednot p.is_absolute()resolves underHERMES_HOMEtest_relative_entry_is_resolved_against_hermes_homep == local_skillsdroptest_local_skills_dir_is_silently_droppedp in seendeduptest_duplicate_entries_collapsep.is_dir()skiptest_nonexistent_entries_are_skippedif not entry: continuetest_empty_and_whitespace_entries_are_skippedTest design
A single
isolated_hermes_homefixture spins up a throwaway~/.hermes/undertmp_path, monkeypatchesHERMES_HOME,HOME, andPath.home, and clears_external_dirs_cache_clear()before and after each case so no test leaks into another. Every test writes its ownconfig.yamlbody — there is no shared YAML fixture to mis-edit later.Verification
Full repo suite (
pytest tests/ -q --ignore=tests/integration --ignore=tests/e2e -n auto) was also run; the new file adds 11 passes and no regressions. The 37 unrelated failures observed in the run reproduce on the parent commit78b0008f4without this patch (mostly env-dependent: anthropic-adapter OAuth, systemd-on-macOS, file-state registry).Out of scope
No production code is touched — this PR is pure test coverage.
Generated by Hermes Turbo
Generated by Hermes Turbo