Add comprehensive tests for ManagedGlossary and McpAnalysisTools

.apm/skills/kyber-weave-docs/SKILL.md

@@ -4,7 +4,7 @@ description: "Generate conformant Kyber-Weave frontmatter for repository documen
 license: MIT
 metadata:
   author: dpalfery
-  version: 0.1.2
+  version: 0.1.3
 ---
 
 # Authoring Kyber-Weave documentation
@@ -188,6 +188,31 @@ Agents and skills should look up the following properties dynamically to find th
 Fix what a rule reports. Do not widen the ontology in `.kyber-weave/kyber-weave.yml` to
 make a failure disappear — that discards the guarantee the corpus exists to provide.
 
+## After conformance: analyze, do not auto-rewrite
+
+When the repository supports documentation analysis, run it only after `docs validate`
+and `docs drift` are clean:
+
+```bash
+kyber-weave docs analyze .
+```
+
+Treat duplicate, conflict, and terminology findings as evidence to review. Never merge,
+delete, or rewrite source documentation merely because analysis paired two claims. Exact
+duplicates are deterministic; conflicts and distinct term senses need a scope-aware human
+or agent verdict through `docs review export` / `docs review import`.
+
+Use `<kyber-ignore>` only for intentional, reviewed cases and only with `duplicate`,
+`conflict`, `terminology`, or `all`. The tags are case-sensitive, balanced, non-nested,
+and cannot cross frontmatter or a `##` boundary. Malformed suppression is an operational
+error rather than a silent ignore.
+
+`docs glossary .` previews terminology proposals; `--write` merges them into the one
+configured glossary without rewriting source documents. The glossary remains a conformant
+`reference` document. Humans approve/reject sense rows, supply approved definitions and
+component/code scopes, update `last-reviewed`, and return the document to `current` after
+review. Do not invent a glossary doc-type or use sense-row status as document status.
+
 ## Never
 
 - Invent a `component` or `owner` that is not in the catalog
@@ -197,3 +222,5 @@ make a failure disappear — that discards the guarantee the corpus exists to pr
 - Change `doc-type` or `status` vocabularies to fit one document
 - Set `status: current` on frontmatter you filled in without review
 - Backdate or forward-date `last-reviewed` — use the date it was actually reviewed
+- Auto-rewrite source prose from an unreviewed duplicate, conflict, or terminology candidate
+- Use ignore markup to hide a finding whose scope or evidence has not been reviewed

.apm/skills/kyber-weave-docs/references/rules.md

@@ -96,3 +96,57 @@ not fail the build.
 `docs validate` and `docs drift` exit non-zero on any **error**. Warnings and info do not
 gate. `--no-info` hides informational findings; `--format sarif` emits SARIF for code
 scanning.
+
+## Analysis — `docs analyze`
+
+Analysis is advisory by default and never edits source documentation.
+
+### `KW-DOC-ANALYSIS-001` — duplicate cluster
+
+Info for a pending near duplicate; Warning for a deterministic exact cluster or a
+high-confidence imported duplicate verdict. Confirm that the claims are substantively the
+same, not merely about the same topic.
+
+### `KW-DOC-ANALYSIS-002` — potential conflict
+
+Info while pending; Error only after a high-confidence imported `conflict` verdict.
+Confirm that both claims cannot be true in the same scope and time before choosing a
+canonical source.
+
+### `KW-DOC-ANALYSIS-003` — ambiguous terminology
+
+Warning when one informative term occurs in divergent contexts not fully accounted for by
+approved scoped glossary senses. Preview proposals with `docs glossary .`; do not rename
+terms automatically.
+
+### `KW-DOC-ANALYSIS-004` — invalid ignore markup
+
+Operational Error. `<kyber-ignore>` must be balanced, case-sensitive, non-nested, use
+`duplicate`, `conflict`, `terminology`, or `all`, and stay within frontmatter/`##`
+boundaries. Fix the markup; suppression never fails open.
+
+### `KW-DOC-ANALYSIS-005` — CodeGraph unavailable
+
+Warning. Analysis continues with document relationships and bounded lexical search. Build
+or restore `.codegraph/codegraph.db` for code-neighborhood evidence.
+
+### `KW-DOC-ANALYSIS-006` — embedding unavailable
+
+Warning in `prefer`, operational Error in `required`. Embeddings remain off by default and
+are never invoked unless the local cache path is safely ignored. Restore the loopback
+provider/safe cache, use `prefer` for lexical fallback, or use `off`.
+
+### `KW-DOC-REVIEW-001` — invalid or stale verdict bundle
+
+Operational Error. Regenerate candidates from the current corpus and validate every
+candidate id, claim hash, evidence id, label, confidence, and glossary proposal. Import is
+atomic; one invalid item writes nothing.
+
+### `KW-DOC-GLOSSARY-001` — invalid managed glossary
+
+Operational Error. Keep the document a conformant `reference`; use only `proposed`,
+`approved`, or `rejected` row status. Approved senses require a definition and at least
+one valid `component:<catalog value>` or `code-ref:<symbol>` scope.
+
+`docs analyze --fail-on none|warning|error` controls finding gating. Operational errors
+always return non-zero.

README.md

@@ -40,16 +40,18 @@ artifact as current guidance, and returns budgeted excerpts that name what they
 [Retrieval →](docs/docgraph/retrieval.md)
 
 ```bash
-kyber-weave docs init .        # scaffold config, catalog, ontology + deploy the authoring skill
+kyber-weave docs init .        # scaffold config/catalog/ontology, protect local cache, deploy the skill
 kyber-weave docs validate .
 kyber-weave docs drift .
+kyber-weave docs analyze .     # advisory duplicate/conflict/terminology findings
 kyber-weave docs graph . --out ./build/doc-graph
 ```
 
 ### Adopting an existing tree
 
 `docs init` does the mechanical half — host config, the catalog that supplies the
-component and owner vocabularies, and the ontology reference every diagnostic points at.
+component and owner vocabularies, the ontology reference every diagnostic points at, and
+the narrow ignored cache path analysis needs before it can persist verdicts or vectors.
 It then deploys the **`kyber-weave-docs` skill** through
 [APM](https://microsoft.github.io/apm), defaulting to `.agents/skills/` so every
 APM-supported client picks it up.
@@ -67,11 +69,35 @@ corpus degrades gracefully rather than serving unreviewed metadata as current gu
 |---|---|
 | `docs_explore(query, maxDocs, charBudget)` | Ranked documents with frontmatter identity, prose within budget, and code joins as `symbol → file:line` |
 | `docs_for_symbol(symbol)` | Reverse lookup: documents whose `code-refs` **formally claim** a symbol — not those that merely mention it |
+| `docs_analysis_candidates(kind, cursor, limit, charBudget)` | Capped, stable, read-only duplicate/conflict/terminology evidence with local cost metrics |
+| `docs_glossary(term)` | Capped, read-only lookup of managed term senses, scopes, and aliases |
 
 It is a separate binary from the CLI on purpose: JSON-RPC owns stdout and Spectre.Console
 also writes there, so separate entry points make stream corruption structurally
 impossible. [MCP runbook →](docs/docgraph/mcp-runbook.md)
 
+### Documentation analysis and terminology
+
+`docs analyze` extracts line-addressable claims from paragraphs, list items, table rows,
+and code fences, then uses DocGraph and one-hop CodeGraph relationships before bounded
+lexical search. Exact duplicates are deterministic; potential conflicts and distinct term
+senses can be exported for agent review and imported as reusable, content-addressed
+verdicts. Source documents are never rewritten.
+
+```bash
+kyber-weave docs analyze .
+kyber-weave docs review export . --out candidates.json
+kyber-weave docs review import . --in verdicts.json
+kyber-weave docs glossary .          # preview
+kyber-weave docs glossary . --write  # merge proposals into one reference document
+```
+
+Embeddings are off by default. When enabled, endpoints must resolve only to loopback,
+redirects are disabled, and no document text is sent unless the local SQLite cache is
+safely ignored. Default hybrid search avoids all-pairs work; `high-recall` is an explicit
+quadratic first pass outside the default latency target.
+[Analysis and review →](docs/docgraph/analysis.md)
+
 ### One external dependency, and it's optional
 
 `docs drift` and `docs graph` resolve symbols against a **CodeGraph** index at
@@ -157,6 +183,7 @@ dotnet test tests/KyberWeave.Tests/KyberWeave.Tests.csproj -c Release
 - **`docs init` expects [APM](https://microsoft.github.io/apm)** to deploy the authoring skill. Both it and CodeGraph are *expected* dependencies — Kyber-Weave detects them and degrades with a message, but never installs anything on your machine.
 - **Security scanning is necessary but not sufficient** — pair it with human review.
 - **The document index is rebuilt, never persisted.** Editing one document rebuilds the whole corpus; comfortable at hundreds of documents, worth revisiting at thousands.
+- **Documentation analysis persistence is a separate local cache.** `.kyber-weave/cache/docs-analysis.sqlite3` stores reusable vectors and verdicts only when the narrow cache path is safely ignored; it is never a source of retrieval prose.
 - Some agent Core APIs exist without CLI verbs (`agent route` / `lint` / `new`) — known gap.
 
 > **Naming note.** "Kyber" collides with CRYSTALS-Kyber / ML-KEM, the NIST post-quantum KEM

docs/README.md

@@ -4,7 +4,7 @@ title: Kyber-Weave documentation
 doc-type: index
 status: current
 owner: dpalfery
-last-reviewed: 2026-08-01
+last-reviewed: 2026-08-12
 ---
 
 # Kyber-Weave documentation
@@ -36,6 +36,7 @@ retrieval graph served to agents over MCP. The primary feature.
 | [Adoption](docgraph/onboarding.md) | `docs init`, the authoring skill, retrofitting an existing tree |
 | [Architecture](docgraph/architecture.md) | The pipeline, the two-clock reload, the code-graph join |
 | [Retrieval and ranking](docgraph/retrieval.md) | Scoring, authority weighting, budgeted excerpts |
+| [Analysis and review](docgraph/analysis.md) | Graph-first duplicate/conflict/terminology detection, agent verdicts, managed glossary |
 | [Governance gates](docgraph/governance.md) | `docs validate`, `docs drift`, `docs catalog` |
 | [MCP server runbook](docgraph/mcp-runbook.md) | Serving the graph to an agent |
 

docs/catalog.md

@@ -4,7 +4,7 @@ title: Component and owner catalog
 doc-type: reference
 status: current
 owner: dpalfery
-last-reviewed: 2026-08-01
+last-reviewed: 2026-08-12
 ---
 
 # Component and owner catalog
@@ -19,7 +19,7 @@ answers for it, and where its source lives.
 
 | Component | Type | Source root | Overview | Detailed documentation | Owner | Last reviewed | Status |
 |---|---|---|---|---|---|---|---|
-| DocGraph | Feature | `src/KyberWeave.Core/Docs` | The opinionated documentation ontology, its conformance gates, and the in-memory retrieval graph served over MCP. | [docgraph/architecture.md](docgraph/architecture.md) | dpalfery | 2026-08-01 | current |
+| DocGraph | Feature | `src/KyberWeave.Core/Docs` | The documentation ontology, conformance gates, graph-first claim analysis, managed terminology, and retrieval graph served over MCP. | [docgraph/architecture.md](docgraph/architecture.md) · [docgraph/analysis.md](docgraph/analysis.md) | dpalfery | 2026-08-12 | current |
 | ContextHygiene | Feature | `src/KyberWeave.Core/Skills` | Governance for the artifacts that shape an agent's context: Agent Skills and harness agent definitions. | [context-hygiene/skills.md](context-hygiene/skills.md) | dpalfery | 2026-08-01 | current |
 | CI Pipelines | Feature | `src/KyberWeave.Core/Diagnostics` | The diagnostic engine every gate reports through: stable rule ids, severity gating, and SARIF. | [ci-pipelines/architecture.md](ci-pipelines/architecture.md) | dpalfery | 2026-08-01 | current |
 | Distribution | Supporting | `scripts` | Self-contained platform binaries and the install path that places them. | [install.md](install.md) | dpalfery | 2026-08-01 | current |

docs/ci-pipelines/rule-reference.md

@@ -5,7 +5,7 @@ doc-type: reference
 status: current
 component: CI Pipelines
 owner: dpalfery
-last-reviewed: 2026-08-01
+last-reviewed: 2026-08-12
 ---
 
 # Rule reference
@@ -34,6 +34,28 @@ baselines — see [CI Pipelines architecture](architecture.md) for why they neve
 | `KW-DOC-DRIFT-002` | Error | `api-endpoints` route matches no indexed route |
 | `KW-DOC-DRIFT-003` | Warning | `source-root` exists but nothing beneath it is indexed |
 
+### Analysis — `docs analyze`
+
+| Id | Severity | Meaning |
+|---|---|---|
+| `KW-DOC-ANALYSIS-001` | Info / Warning | Duplicate cluster. Pending near duplicates inform; exact or high-confidence confirmed duplicates warn. |
+| `KW-DOC-ANALYSIS-002` | Info / Error | Potential conflict. Only a high-confidence imported conflict verdict errors. |
+| `KW-DOC-ANALYSIS-003` | Warning | Ambiguous terminology not fully explained by approved scoped senses. |
+| `KW-DOC-ANALYSIS-004` | Operational Error | Malformed, nested, unknown, or cross-boundary ignore markup. |
+| `KW-DOC-ANALYSIS-005` | Warning | CodeGraph unavailable; document relationships and bounded lexical search continue. |
+| `KW-DOC-ANALYSIS-006` | Warning / Operational Error | Embeddings unavailable: warning in `prefer`, error in `required`. |
+
+### Review and managed glossary
+
+| Id | Severity | Meaning |
+|---|---|---|
+| `KW-DOC-REVIEW-001` | Operational Error | Verdict bundle is invalid/stale, or safe atomic persistence is unavailable. |
+| `KW-DOC-GLOSSARY-001` | Operational Error | Configured managed glossary has invalid structure, status, definition, or scope. |
+
+Analysis findings respect `docs analyze --fail-on`; operational errors always return
+non-zero. See [analysis and review](../docgraph/analysis.md) for classifier and lifecycle
+details.
+
 ## Skills — [Skill governance](../context-hygiene/skills.md)
 
 | Id range | Tier | Meaning |

docs/configuration.md

@@ -4,7 +4,7 @@ title: Configuration
 doc-type: reference
 status: current
 owner: dpalfery
-last-reviewed: 2026-08-04
+last-reviewed: 2026-08-12
 code-refs:
   - KyberWeaveConfigLoader
   - OntologyConfig
@@ -55,6 +55,30 @@ harness:
       supports-native-parent-agents: false
       mapped-role-skill-overrides:
         reviewer: code-review
+
+docs-analysis:
+  statuses: [current]
+  glossary-path: docs/glossary.md
+  verdict-confidence: 0.80
+  search:
+    mode: hybrid
+    min-claim-tokens: 5
+    lexical-candidate-threshold: 0.45
+    lexical-duplicate-threshold: 0.90
+    semantic-candidate-threshold: 0.78
+    semantic-duplicate-threshold: 0.92
+    terminology-context-threshold: 0.30
+    max-neighbors-per-claim: 10
+    max-code-neighbors: 50
+    max-candidates: 500
+  embeddings:
+    mode: off
+    endpoint: http://127.0.0.1:1234/v1/embeddings
+    model: configured-model-name
+    dimensions: 768
+    batch-size: 64
+    timeout-seconds: 60
+    api-key-env: LOCAL_EMBEDDING_TOKEN
 ```
 
 ## Ontology keys
@@ -135,6 +159,58 @@ returns one has actively misled its caller. This pairs with the authority weight
 [retrieval](docgraph/retrieval.md), which demotes plans and superseded documents that are
 still in scope.
 
+## Documentation analysis
+
+The entire `docs-analysis` section is optional. Presets establish the defaults below; the
+individual values are advanced overrides rather than a required tuning exercise.
+
+| Key | Default | Effect |
+|---|---|---|
+| `statuses` | `[current]` | Existing ontology statuses eligible for claim extraction |
+| `glossary-path` | `<first-docs-root>/glossary.md` | Managed glossary, always beneath a configured docs root |
+| `verdict-confidence` | `0.80` | Minimum imported confidence for a durable classification or suppression |
+| `search.mode` | `hybrid` | `graph`, bounded `hybrid`, or explicitly expensive `high-recall` |
+| `search.min-claim-tokens` | `5` | Claims below this token count are not compared |
+| `search.lexical-candidate-threshold` | `0.45` | Minimum lexical evidence for ordinary candidacy |
+| `search.lexical-duplicate-threshold` | `0.90` | Lexical near-duplicate threshold |
+| `search.semantic-candidate-threshold` | `0.78` | Minimum semantic evidence for candidacy |
+| `search.semantic-duplicate-threshold` | `0.92` | Semantic near-duplicate threshold |
+| `search.terminology-context-threshold` | `0.30` | Maximum contextual similarity for divergent senses |
+| `search.max-neighbors-per-claim` | `10` | Per-source top-k bound |
+| `search.max-code-neighbors` | `50` | Code nodes above this degree are non-discriminating and skipped |
+| `search.max-candidates` | `500` | Hard analysis/review candidate cap |
+
+`graph` compares global exact duplicates plus graph neighbors. `hybrid` adds a sparse
+corpus-wide inverted-index fallback without an all-pairs scan and is the default.
+`high-recall` broadens lexical comparison and, when embeddings are enabled, performs a
+global exact cosine first pass. That first pass is explicitly quadratic and is outside
+the 10-second default-path target. See [analysis and review](docgraph/analysis.md).
+
+### Embeddings are local, optional, and persistence-gated
+
+`embeddings.mode` is one of:
+
+| Mode | Behavior |
+|---|---|
+| `off` | Default. Never constructs or invokes an embedding provider. |
+| `prefer` | Uses cached/local embeddings when safe; warns and falls back to lexical analysis otherwise. |
+| `required` | Treats an unavailable provider or unsafe cache as an operational error. |
+
+When mode is `prefer` or `required`, `endpoint` and `model` are required. `dimensions` is
+optional; the compatible request always sends batched string input, the model, and
+`encoding_format: float`. `batch-size` defaults to 64 and `timeout-seconds` to 60.
+`api-key-env` names an environment variable; it is not the token itself.
+
+The endpoint must be an absolute HTTP(S) URI whose every resolved address is loopback:
+`localhost`, the full `127.0.0.0/8` range, or `::1`. Kyber-Weave validates again when the
+socket connects and disables redirects, so a local name or response cannot escape to a
+remote endpoint. Credentials and headers are not logged or persisted.
+
+Embedding calls are also gated by `.kyber-weave/.gitignore` effectively protecting the
+narrow `cache/` path and by the cache not already being tracked. Without that proof,
+Kyber-Weave sends no document text. `prefer` falls back; `required` fails. `docs init`
+safely merges the ignore entry for new and existing hosts.
+
 ## Harness profiles
 
 Harnesses differ in what they can express, so [agent parity](context-hygiene/agents.md)