docs(#138): track mechanical-consistency autonomy class for review agent

docs/problems/applied/konflux-ci/README.md

@@ -100,6 +100,54 @@ The Security review sub-agent covers two Konflux-specific concerns within a sing
 - Release policy — can release gates be bypassed?
 - Artifact integrity — can artifacts be tampered with?
 
+### Autonomy readiness evidence
+
+Evidence tracking for [autonomy classes](../../autonomy-spectrum.md#evidence-driven-autonomy-classes) observed in konflux-ci repos. Each class has qualifying criteria, observed evidence, and a validation plan.
+
+#### Mechanical consistency in the architecture repo
+
+The [konflux-ci/architecture](https://github.com/konflux-ci/architecture) repo is documentation-only (ADRs, service overviews, architecture diagrams). It qualifies for the mechanical-consistency autonomy class described in the [general problem doc](../../autonomy-spectrum.md#mechanical-consistency-in-documentation-repos).
+
+**Observed evidence (PR #367, Jun–Jul 2026):**
+
+[PR #367](https://github.com/konflux-ci/architecture/pull/367) was reviewed by 5 human reviewers (rcerven, johnbieren, cit1zen, ralphbean, arewm) and the review agent (7 runs between Jun 11 and Jul 6). The review agent produced 4 genuine findings:
+
+| # | Finding | Severity | Category | Human-caught? | Addressed? |
+|---|---|---|---|---|---|
+| 1 | AGENTS.md listed ADR status as 'Pr' but frontmatter said 'Accepted' | HIGH | Mechanical (status consistency) | No | Yes — author fixed |
+| 2 | 'occurances' should be 'occurrences' | LOW | Mechanical (typo) | No | Yes — author fixed |
+| 3 | Phase 3 action items skip from 2 to 4 | MEDIUM | Mechanical (numbering gap) | No | No — merged without fix |
+| 4 | `image.redhat.com/*` in Context but Decision criteria only cover 'appstudio' prefixes | MEDIUM | Mechanical (scope coherence) | No | No — merged without fix |
+
+The agent also raised a domain-incorrect finding about Kubernetes label selector immutability. Human reviewer mmorhun dismissed it: "We don't use such resources." This validates the autonomy boundary — the agent reliably catches mechanical consistency issues but applies generic domain knowledge incorrectly when project-specific context is needed.
+
+Human reviewer cit1zen caught an ADR number collision requiring renumbering — a cross-PR context issue the agent cannot currently detect (separate capability gap, not a mechanical consistency failure).
+
+**What the evidence shows:**
+
+- The agent caught 2 medium-severity mechanical consistency issues that 5 human reviewers missed, and both merged without fix
+- The missed findings (numbering gap, scope coherence) are exactly the kind of pattern-matchable, document-internal checks where agents have a systematic advantage
+- COMMENT-level findings were insufficient to prevent merge of medium-severity issues — CHANGES_REQUESTED would have increased the likelihood of these being addressed
+
+**Validation plan:**
+
+Track the next 5 ADR PRs merged in konflux-ci/architecture. For each PR, record:
+
+1. Review agent mechanical consistency findings (typos, numbering, cross-references, scope coherence)
+2. Human reviewer comments on mechanical consistency
+3. Whether unresolved medium+ mechanical findings would have been addressed with CHANGES_REQUESTED
+
+Success criteria: the agent catches mechanical consistency issues that no human reviewer flags on at least 3 of 5 PRs. A case where a human catches a mechanical consistency issue the agent missed weakens the signal.
+
+| PR | Agent mechanical findings | Human mechanical findings | Unresolved medium+? | Notes |
+|---|---|---|---|---|
+| #367 (baseline) | 4 (2 HIGH/LOW fixed, 2 MEDIUM unfixed) | 0 mechanical | Yes (2) | Evidence PR |
+| | | | | |
+| | | | | |
+| | | | | |
+| | | | | |
+| | | | | |
+
 ### Repo readiness
 
 Data from the [coverage dashboard](https://konflux-ci.dev/coverage-dashboard/) (as of March 2026):

docs/problems/autonomy-spectrum.md

@@ -93,6 +93,57 @@ This addresses the gap where the binary model can miss risky changes that don't
 - The dimensions listed above are examples, not necessarily exhaustive. Different organizations might weight or define them differently.
 - Could produce false escalations (agent is uncertain, so it escalates conservatively) or false confidence (agent misjudges blast radius). Shadow mode data would help calibrate.
 
+## Evidence-driven autonomy classes
+
+The binary per-repo model and per-decision escalation dimensions above describe autonomy in broad strokes. In practice, specific categories of agent action accumulate evidence that they can be trusted at a higher autonomy level before the repo as a whole graduates. These are **autonomy classes** — narrow, well-defined categories of agent behavior where empirical evidence supports granting the agent more authority.
+
+Each autonomy class is defined by:
+
+1. **Qualifying criteria** — what properties a change must have to fall into this class
+2. **Evidence** — observed cases where the agent outperformed or matched human review
+3. **Validation plan** — how to confirm the pattern holds before changing autonomy level
+4. **Proposed autonomy change** — what the agent would do differently once validated (e.g., CHANGES_REQUESTED instead of COMMENT)
+5. **Boundary conditions** — what the agent should *not* do even within this class
+
+Autonomy classes are distinct from repo-level graduation. A repo that is not autonomous can still have specific autonomy classes where the review agent operates at a higher level — as long as the class boundaries are narrow enough that false positives are acceptable and domain-specific judgment is not required.
+
+### Mechanical consistency in documentation repos
+
+**Qualifying criteria:**
+
+- Target repo is documentation-only (no application code)
+- Finding category is mechanical consistency: typos, numbering gaps, cross-reference validation, scope coherence between document sections
+- Finding does not require project-specific domain knowledge to evaluate
+
+**Evidence:** See [applied docs](applied/) for the specific tracking data. In the baseline observation, a review agent caught mechanical consistency findings (numbering gaps, scope ambiguity between document sections) that multiple human reviewers missed — these merged without fix. The agent also caught a status inconsistency and a typo that the author fixed. However, the agent produced one domain-incorrect finding that a human reviewer dismissed because the project does not use the resource types in question.
+
+**Boundary conditions:** The domain-incorrect finding demonstrates a clear autonomy boundary. Mechanical consistency checks (pattern-matchable, document-internal) are within the agent's reliable capability. Domain-specific architectural judgments (requiring knowledge of which technologies the project actually uses) are not.
+
+**Proposed autonomy change:** Options for increasing review authority on validated mechanical consistency findings:
+
+1. **CHANGES_REQUESTED for medium+ mechanical findings** — the review agent blocks merge until the author addresses mechanical consistency findings at medium severity or above. Domain-specific findings remain at COMMENT level. This is the most direct response to the evidence (medium-severity findings merging without fix under COMMENT).
+2. **Auto-file tracking issues for unresolved findings** — instead of blocking merge, the agent files a tracking issue when medium+ mechanical findings go unresolved. Lower friction than CHANGES_REQUESTED but doesn't prevent the problem (findings still merge without fix).
+3. **COMMENT with explicit acknowledgment prompt** — keep COMMENT but require the author to explicitly acknowledge each medium+ finding (dismiss or address). This is intermediate friction — less blocking than CHANGES_REQUESTED but more than the current silent merge.
+
+Option 1 is the strongest match for the evidence (COMMENT was insufficient to prevent merge of medium-severity issues), but requires the validation plan to succeed first. Options 2 and 3 are lower-risk alternatives that could be adopted with less evidence.
+
+**Validation plan:** Track the next 5 ADR PRs merged in the qualifying repo. For each, compare agent mechanical consistency findings against human reviewer comments. Success: the agent catches mechanical consistency issues that no human flags on at least 3 of 5 PRs. A case where a human catches a mechanical consistency issue the agent missed weakens the signal. See [applied/konflux-ci](applied/konflux-ci/) for the specific tracking data.
+
+**Cross-references:**
+
+- Distinct from the bot-dependency-bump autonomy class (where review value comes from security verification, not consistency checking)
+- The scope-ambiguity finding is evidence for auto-filing tracking issues for unresolved medium+ findings
+- The ADR number collision caught by a human reviewer (but missed by the agent) was a cross-PR context issue — a separate capability gap, not a mechanical consistency failure
+
+### Other potential autonomy classes
+
+Additional autonomy classes may emerge from operational evidence. Examples under observation:
+
+- **CI-config-only changes** — trivial workflow file modifications where human reviewers consistently approve in under 5 minutes with no substantive comments
+- **Bot dependency bumps** — automated version bumps where review value is security verification rather than code quality
+
+Each class requires its own evidence trail and validation plan before any autonomy change.
+
 ## Open questions
 
 - Who decides when a repo is ready for autonomy? (See [governance.md](governance.md))

docs/problems/code-review.md

@@ -205,3 +205,4 @@ Whether this belongs in the review system or is a separate workflow operating on
 - How do we handle multi-repo changes where the review needs to consider changes across repos together?
 - Is there a "coherence" or "fitness" review dimension that can't be decomposed into the existing sub-agents? The [taste problem](contribution-volume.md#the-taste-problem) suggests that some review concerns require holistic project judgment rather than specialized analysis.
 - At what volume does the salvage model become cost-effective compared to rejection? What's the token cost of salvaging a typical contribution vs. the contributor-retention value?
+- Should review agents use CHANGES_REQUESTED instead of COMMENT for [mechanical-consistency findings](autonomy-spectrum.md#mechanical-consistency-in-documentation-repos) where they demonstrably outperform human reviewers?