Conversation
Adds a structural linter for SKILL.md files, complementing `skills audit` (security) and the hub update checker. It surfaces problems the runtime otherwise hides: lenient YAML fallback, silently dropped env-var entries, and silent name/description truncation. - tools/skills_lint.py: data-driven rule registry (HSL001-HSL060, 18 rules), lint_skill(), format_lint_report(), installed-skill enumeration helpers. Structural failures suppress field rules to avoid misleading cascades. - agent/skill_utils.py: parse_frontmatter_strict() - surfaces YAML errors instead of falling back to naive key:value parsing. - hermes_cli: `skills lint [targets ...] [--all] [--json] [--fail-on]` with CI-friendly exit codes (0/1/2); do_publish() now lints before the security scan, so lint errors block publishing. - Docs: full rule reference, severity/exit-code semantics, CI guidance. - Tests: 74 passing (per-rule unit tests + CLI exit codes, --json, --fail-on, publish-blocks-on-error). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
teknium1
left a comment
There was a problem hiding this comment.
Thanks for the structural-validator contribution. The current runtime still has the lenient YAML fallback at agent/skill_utils.py:145-155, and current main has no equivalent lint command, so the premise is valid.
Problems
tools/skills_lint.py:53omitstermuxandandroid, althoughagent/skill_utils.py:183explicitly accepts both under Termux. HSL020 would falsely emit a blocking error, including during the proposed publish gate athermes_cli/skills_hub.py:1391-1396.website/docs/user-guide/features/skills.md:714describes--allas repository-wide CI linting, buthermes_cli/skills_hub.py:1051-1056only enumerates installed profile/external skill paths.
Suggested changes
- Align HSL020 with the runtime platform matcher and add Termux/Android regression tests.
- Correct the CI guidance, or implement and test an explicit recursive repository/root lint mode.
Automated hermes-sweeper review.
| } | ||
| ) | ||
|
|
||
| VALID_PLATFORMS = frozenset({"linux", "macos", "windows"}) |
There was a problem hiding this comment.
agent/skill_utils.py:183 explicitly accepts termux and android platform tags in a Termux session. Excluding both here makes HSL020 a false blocking error and can cause the new publish gate to reject a runtime-supported skill; include them or share the runtime's accepted-tag definition.
|
|
||
| ```bash | ||
| # Fail the build on any structural error in any skill under this repo | ||
| hermes skills lint --all || exit 1 |
There was a problem hiding this comment.
--all uses find_installed_skill_paths(), which scans the active profile and configured external skill directories, not this repository. This command does not guarantee that repo skills are linted in CI; revise the example or add a repository-root recursive mode.
|
Heads-up: #115176 wires the linter that merged in #81896 as |
What
Adds
hermes skills lint, a structural validator forSKILL.mdfiles. It complements the existinghermes skills audit(security scanning) and the hub update checker:auditasks "is this skill dangerous?",lintasks "is this skill well-formed?".Why
Hermes is deliberately forgiving at runtime — when a skill's frontmatter is malformed, it degrades quietly rather than crashing. That is good for end users but bad for skill authors, who never learn their skill is subtly broken. Today:
parse_frontmatter()silently falls back to naivekey:valueparsing on broken YAML, dropping nested fields._get_required_environment_variables()silently drops malformed env-var entries.name/descriptionvalues are silently truncated.skills publishonly checked that adescriptionexists and that the security scan passes.lintsurfaces exactly those hidden problems, with stable rule IDs and CI-friendly exit codes.What changed
tools/skills_lint.py(new) — data-driven rule registry (18 rules,HSL001–HSL060),lint_skill(),format_lint_report(), and installed-skill enumeration helpers. Structural failures (e.g. broken YAML) suppress the field rules so authors do not get a misleading "name missing" cascade on top of the real error.agent/skill_utils.py— addsparse_frontmatter_strict(), which surfaces YAML errors instead of papering over them. The existing lenientparse_frontmatter()is untouched.hermes_cli/— new subcommandhermes skills lint [targets ...] [--all] [--json] [--fail-on error|warning]with exit codes0(clean) /1(findings at/above threshold) /2(usage error).do_publish()now runs lint before the security scan; lint errors block publishing, warnings do not.website/docs/user-guide/features/skills.mdgains a full rule reference, severity/exit-code semantics, and CI guidance.Severity model
Testing
tests/tools/test_skills_lint.py(including a test that pins the gap — the same broken YAMLHSL002flags is swallowed without error by the lenient runtime parser), plus CLI tests for exit codes,--json,--fail-on, and publish-blocks-on-lint-error intests/hermes_cli/.prerequisitesblocks, name/dir mismatches, missing referenced files).Example
🤖 Generated with Claude Code