feat(observability): aggregate bounded skill metrics

[afourniernv]

What does this PR do?

Adds bounded skill lifecycle and reuse metrics on top of the tool-metrics stack. Successful Hermes skill operations emit privacy-safe Relay marks that aggregate into lifecycle and load counters without exporting skill names, paths, exact use counts, or patch generations.

This is stacked on #68882. Until the earlier PRs land, GitHub shows the cumulative stack against main; the skill-only diff is available here.

Stack Order

The core Relay integration and export fixes have landed in #67607, #73120, and #73544.

1. feat(observability): report model and provider usage #68881 - bounded model usage, latency, token, retry, cost, family, and name metrics.
2. feat(observability): aggregate bounded tool metrics #68882 - bounded tool lifecycle and approval metrics.
3. This PR - skill lifecycle, provenance, reuse, and reuse-after-patch metrics.
4. feat(observability): add Relay client resource metrics #68978 - bounded client resource dimensions.
5. feat(observability): add Relay active install metrics #69416 - active-install metrics.
6. feat(observability): add Relay setup and first-use metrics #69437 - setup and first-use metrics.

Merge and review in this order.

Related Issue

Depends on #68882.

Type of Change

• 🐛 Bug fix (non-breaking change that fixes an issue)
• ✨ New feature (non-breaking change that adds functionality)
• 🔒 Security fix
• 📝 Documentation update
• ✅ Tests (adding or improving test coverage)
• ♻️ Refactor (no behavior change)
• 🎯 New skill (bundled or hub)

Changes Made

• Emit successful install, create, edit, patch, archive, stale, and restore facts from their authoritative Hermes operations.
• Derive first use, reuse, and one-time reuse-after-patch state transactionally in the existing skills/.usage.json store.
• Add closed Relay mark contracts and SQLite/package counters for hermes.skill.lifecycle.count and hermes.skill.load.count.
• Keep raw skill identity and exact continuity state local; shared-metrics events and packages contain only allowlisted dimensions and buckets.
• Correlate task-owned skill facts only to the exact active task, rejecting stale or explicitly mismatched task events instead of counting them globally.
• Reset stale ownership and continuity when a new skill reuses an old name, and recover malformed local counter state without suppressing later reuse-after-patch facts.
• Keep the native smoke compatible with the once-per-day production export gate while force-packaging pending test deltas for immediate schema and privacy validation.

How to Test

1. Run uv run pytest -q tests/tools/test_skill_usage.py tests/tools/test_skill_manager_tool.py tests/hermes_cli/test_relay_shared_metrics.py tests/hermes_cli/test_relay_shared_metrics_runtime.py tests/agent/test_skill_commands.py tests/agent/test_skill_bundles.py tests/cron/test_scheduler.py tests/tools/test_skills_tool.py tests/tools/test_skills_hub.py tests/hermes_cli/test_skills_hub.py tests/test_plugin_skills.py.
2. Run uv run ruff check tools/skill_usage.py hermes_cli/observability/relay_shared_metrics.py scripts/smoke_nemo_relay_shared_metrics.py tests/tools/test_skill_usage.py tests/hermes_cli/test_relay_shared_metrics_runtime.py.
3. With the published nemo-relay==0.6.0 binding installed, run .venv/bin/python scripts/smoke_nemo_relay_shared_metrics.py.

Checklist

Code

• I've read the Contributing Guide
• My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
• I searched for existing PRs to make sure this isn't a duplicate
• My PR contains only changes related to this fix/feature (no unrelated commits)
• I've run pytest tests/ -q and all tests pass
• I've added tests for my changes (required for bug fixes, strongly encouraged for features)
• I've tested on my platform: macOS arm64

Documentation & Housekeeping

• I've updated relevant documentation (README, docs/, docstrings) — or N/A
• I've updated cli-config.yaml.example if I added/changed config keys — or N/A
• I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — or N/A
• I've considered cross-platform impact (Windows, macOS) per the compatibility guide — or N/A
• I've updated tool descriptions/schemas if I changed tool behavior — or N/A

Screenshots / Logs

• Affected skill, Relay, tool, scheduler, hub, plugin, and command suites: 1,145 passed, 4 skipped. Two existing _send_to_platform resource warnings remain outside this diff.
• Eight-process continuity probe: 800 exact loads, one first-use fact, and one reuse-after-patch fact after a concurrent load wave.
• Source/schema parity probe: lifecycle, provenance, reuse, and post-patch enums match exactly.
• Real Hermes CLI smoke with published nemo-relay==0.6.0: passed with schema-valid delta packages; model, task, tool, lifecycle, and reuse counters persisted, and both private skill-name canaries were absent.
• The same native smoke also passed against the locally built Relay 0.7.0 binding.

[afourniernv]

CI note: the failing Python slices are unrelated Photon failures already present on current main. I reproduced all six failures directly at origin/main (tests/plugins/platforms/photon/test_inbound.py: 4 failures; test_runtime_record.py: 2 failures), and this stack has no diff under the Photon implementation or tests. The focused upstream fix is #73762, whose full CI is green. The skill-metrics validation listed in this PR body remains green.

[teknium1]

Thanks for the bounded, privacy-focused implementation. The requested skill-metrics premise remains unmet on current main: hermes_cli/observability/shared_metrics_contract.py:17-19 and :123-147 register only model and task counters, while repository-wide searches found no existing skill lifecycle/load metrics or reuse-after-patch state. The PR adds transactional local state before lifecycle emission (tools/skill_usage.py:860-904 in PR head) and coverage for the package path (tests/hermes_cli/test_relay_shared_metrics_runtime.py:3617).

The current stack is not cleanly mergeable: #68881, #68882, and this PR all report DIRTY, so salvage will require resolving the ordered stack against current main.

Automated hermes-sweeper review.

[afourniernv]

Addressed the sweeper mergeability note.

#68881 has now merged, and #68882 is already caught up to that main. I merged the current tool-metrics head into this branch in d20debd44, preserving the stack history. The only conflict was in tests/hermes_cli/test_relay_shared_metrics_runtime.py; the resolution retains the skill lifecycle regression coverage alongside the newer tool/main tests.

Local validation:

• Ruff passed for the resolved runtime test.
• Focused shared-metrics and skill suite: 197 passed.
• Related skill, hub, command, and scheduler suite: 210 passed with two existing unawaited-coroutine warnings.

GitHub now reports the PR mergeable and zero commits behind main; required checks and review remain pending.