feat(B-0058): alignment-clause coverage audit tool#2102
Merged
Conversation
New TS script at tools/alignment/audit_clause_coverage.ts — scans .claude/skills/*/SKILL.md and .claude/agents/*.md for references to the 21 alignment clauses (HC-1..HC-7, SD-1..SD-9, DIR-1..DIR-5) from docs/ALIGNMENT.md. Reports per-surface clause citations and flags surfaces with zero coverage. This is slice 1 of B-0058 (responsibility #2: new-surface audit). Detection surface only — does not block commits. Baseline: 3/260 surfaces cite any clause. 8 clauses uncited anywhere (HC-3, HC-5, SD-3..SD-5, SD-7, SD-9, DIR-3). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
There was a problem hiding this comment.
Pull request overview
Adds a new alignment auditing tool to measure how well skills and agent definitions cite the HC/SD/DIR clauses from docs/ALIGNMENT.md, enabling coverage reporting and optional advisory gating.
Changes:
- Introduces
tools/alignment/audit_clause_coverage.tsto scan.claude/skills/*/SKILL.mdand.claude/agents/*.mdfor clause references and emit human/JSON/Markdown reports (plus--outartifacts). - Adds an optional
--gate Nadvisory threshold to exit non-zero when surfaces fall below the minimum citation count. - Registers the new tool in
tools/alignment/README.md.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| tools/alignment/README.md | Adds the new audit script to the alignment tools registry table. |
| tools/alignment/audit_clause_coverage.ts | Implements clause extraction, surface enumeration, report emission, and optional gating. |
- Remove persona name from code comment (use role-ref only) - Add eslint-disable-line for sonarjs/no-os-command-from-path - Handle non-zero git exit status in repoRoot() - Eliminate non-null assertion via local binding Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This was referenced May 8, 2026
AceHack
added a commit
that referenced
this pull request
May 8, 2026
Adds test coverage for the two alignment tools shipped in PRs #2102-2105: - audit_clause_coverage.test.ts (26 tests): extractClauses regex logic, ALL_CLAUSES constant, audit() integration, main() CLI arg parsing + gate - audit_clause_drift.test.ts (8 tests): main() CLI arg parsing, no-drift baseline (HEAD vs HEAD), default base ref Exports extractClauses and ALL_CLAUSES from audit_clause_coverage.ts to enable direct unit testing of the core regex matching logic. Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
tools/alignment/audit_clause_coverage.ts— a new TS tool that scans all skills (.claude/skills/*/SKILL.md) and agents (.claude/agents/*.md) for references to the 21 alignment clauses (HC-1..HC-7, SD-1..SD-9, DIR-1..DIR-5) fromdocs/ALIGNMENT.md.--json,--md,--out DIR, and--gate Noutput modes matching the sibling tools' conventions.tools/alignment/README.md.This is slice 1 of B-0058 (responsibility #2: new-surface alignment-clause consistency check). Detection surface only — does not block commits or PRs. The
--gateflag is advisory.Focused checks
dotnet build -c Release→ 0 Warning(s), 0 Error(s) ✓bun tools/alignment/audit_clause_coverage.ts→ runs clean, exit 0 ✓bun tools/alignment/audit_clause_coverage.ts --json→ valid JSON output ✓bun tools/alignment/audit_clause_coverage.ts --gate 1→ exit 1 (correctly trips) ✓bun tools/alignment/audit_clause_coverage.ts --help→ prints usage, exit 0 ✓bun tools/alignment/audit_clause_coverage.ts --bogus→ prints error, exit 2 ✓Baseline snapshot
Test plan
bun tools/alignment/audit_clause_coverage.tsruns clean on CI--jsonoutput parses as valid JSON--gate 1exits with code 1--outcreatesclause-coverage.json+clause-coverage.md🤖 Generated with Claude Code