diff --git a/docs/taxonomy/aif-structural-audit-method.md b/docs/taxonomy/aif-structural-audit-method.md new file mode 100644 index 00000000..548cb9e0 --- /dev/null +++ b/docs/taxonomy/aif-structural-audit-method.md @@ -0,0 +1,126 @@ +# AIF structural audit — method & governance record + +> **Provenance.** This document records a design conversation between **jsboige** +> (author of the fallacy taxonomy, 5+ years of incremental work) and the **ai-01 +> coordinator**, 2026-07-22, on how the AIF/Walton mechanistic lens (chantier +> [#498](https://github.com/ArgumentumGames/Argumentum/issues/498)) should relate +> to the *authored structure* of the fallacy taxonomy. It is committed to the repo +> deliberately so the reasoning survives context/dashboard condensation. It is a +> **method + decision record**, not ratified policy — wording is open to jsboige's +> revision. Tracking issue: see the "Actionable" section. + +--- + +## 1. Why this exists + +The AIF Layer-C chantier (#498) annotates each fallacy with an **attack-type** +(`undercut` → RA-node, `undermine` → I-node, `rebut` → CA-node — see +`docs/ontology/aif-export/README.md`). Cluster [#845](https://github.com/ArgumentumGames/Argumentum/pull/845) +was the **first** to cross from *serializing* the taxonomy (annotating what is +there) to *qualifying* it (finding that a node's grouping does not hold under the +mechanistic lens — 6 leaves flagged as out-of-mechanism, **flagged not +force-fitted**). + +That inflection raised jsboige's real question: **are we entering a phase where +the detail of the authored taxonomy becomes criticisable, and structural +adjustments get proposed?** The answer is *yes, partly* — and this document +frames how to do it **fairly to the tree**, without letting an automated lens +flatten five years of deliberate nuance. + +## 2. Two axes of arbitrariness (jsboige's framing) + +The taxonomy carries **two distinct kinds of arbitrariness**, and only one is +revisable: + +| Axis | What | Status | +|------|------|--------| +| **A — the groupings** | The tree structure jsboige built: consolidating clusters, moving grappes, integrating large blocks (codex of cognitive biases, the mental-manipulation / psychological-games grappe). Built with, in his own words, *"as much rigour as I could, i.e. relatively little if I'm honest."* One explicit criterion, kept since the card game: **balancing the tree** — necessarily arbitrary, essentially pedagogical. | **Revisable** | +| **B — the base material** | The fallacies *themselves* are heterogeneous objects. Some carry Latin names and 2000 years of scholastic history (*petitio principii*); others are decades-old coinages for modern phenomena (*whataboutisme*, ~30 yrs). The tree is simultaneously **a classification of argument structures AND a museum of the history of how humans named their bad arguments.** No protocol makes these commensurable. | **Irreducible** | + +**The AIF attack-type is a lens orthogonal to axis A.** A leaf can be correctly +placed (pedagogically) under a node *and* have a different attack-mechanism than +its siblings. **"Misgrouped under the AIF lens" ≠ "misplaced in the taxonomy."** + +## 3. Preliminary read (ai-01 impression — to be replaced by measurement) + +Not a verdict, an **impression** grounded in artifacts, explicitly awaiting the +audit below: + +- **The tree is not decorative. Intuitions sound; changes at the margin.** +- **Strongest evidence:** it took **7 Layer-C clusters** before the first + mechanism-heterogeneity surfaced (#845). Had the tree been arbitrary, + heterogeneity would appear at cluster 1–2 (attack-type distributed at random + under the nodes). The first six came out mechanism-homogeneous — even where a + *scheme* token was missing (fail-loud), the *attack-type* stayed consistent — + although jsboige did **not** build them on the AIF axis. A pedagogical tree that + turns out *also* mechanism-coherent means intuition supplied the formal rigour + that was not consciously applied. + *(Qualified: the "7 clusters / first heterogeneous" framing is RAPPORTÉ by the + chantier and consistent with the merge sequence; the audit turns it into a number.)* +- **Where it will bite:** jsboige's **tree-balancing criterion** — the one + criterion genuinely orthogonal to mechanism. Wherever a grappe was split or + moved to balance branch sizes (card-game constraint), AIF may want to regroup or + re-split. The *"painful arbitrations"* concentrate **there**, plus on **hub + nodes** (e.g. *homme de paille*, central from several angles — also a hub in the + #838 84-pair near-dup set). Not everywhere — on those specific points. + +## 4. The instrument — two-layer homogeneity audit + +The point of the audit is to **replace impression with a per-node number**, and to +be **fair to the tree** by separating tree-tension from material-resistance. + +For each parent node, compute two independent layers: + +- **(a) Attack-type heterogeneity** — distribution of `undercut`/`undermine`/`rebut` + across the node's leaves (a homogeneity ratio or Shannon entropy). **High (a) = + potential tree tension** (the node mixes attack-mechanisms). +- **(b) Fail-loud rate** — fraction of the node's leaves with **no native AIF + scheme token** (the material resisted mechanistic modelling). This is an + *imperfect proxy* for **material resistance** (axis B intruding). The chantier + already records this per leaf (`native token` vs `fail-loud`, e.g. #839). + +**Decision rule:** + +| (a) attack-type heterogeneity | (b) fail-loud rate | Reading | Action | +|---|---|---|---| +| high | low | Genuine **tree tension** — the grouping mixes mechanisms and the material is tractable | Arbitrage worth it (jsboige) | +| any | high | **Material resistance** — axis B, the atoms are just refractory here | Leave the pedagogy alone; annotate the seam, don't regroup | +| low | low | Node is mechanism-coherent | No action — evidence the grouping tracks mechanism | + +This makes the audit **only bill jsboige for what is genuinely his** (grouping +choices), never for the irreducible messiness of the source material. + +**Live specimen already on the table:** *tu quoque* (ancient, Latin) and +*whataboutisme* (modern) — the **same pragmatic move at two eras** — surfaced as +the `rebut`/CA vs `undermine`/I modelling tension in #845. The ancient/modern seam +(axis B) reads *directly* in a modelling tension, which is exactly why layer (b) is +needed to avoid misreading it as an axis-A defect. + +## 5. Governance principle (decision record) + +1. **AIF may *qualify* (critique) the structure — but findings are FLAGS, never + auto-reorganisations.** Workers surface "mechanism-heterogeneity here"; jsboige + alone decides whether it means anything. +2. **All findings gated on jsboige ratification. 0 prod-CSV write** until ratified + (post-tag regime, #499 staging method — accumulate in `docs/taxonomy/`, batch + ratify, then apply). +3. **The pedagogical-balance criterion is legitimate**, orthogonal to mechanism. + Heterogeneity under AIF is **not automatically a defect**. +4. **Preserve rich taxonomy nuance** (jsboige's standing value): where + heterogeneity is material-driven (layer b high), **keep the pedagogical grouping + and annotate the seam** — do not flatten a multi-axis tree onto one axis. + +## 6. Status & actionable + +- **The instrument is half-built — it *is* the AIF chantier (#498).** Current + coverage: 145 fully-modeled fallacies + ~7 Layer-C clusters. The full-tree + per-node number waits for Layer-C coverage to advance. +- **Down-payment ("acompte") available now:** a preliminary homogeneity read on + the already-annotated material (145 fully-modeled + the merged clusters) — a + first per-family homogeneity rate + the visible heterogeneous nodes — enough for + jsboige to sanity-check the "changes at the margin" impression on data. +- Read-only, staged, feeds the post-tag batch ratification. Companion to #497 + (crosslinks), #498 (AIF modelling), #790 (Layer-C pilot design). + +**Tracking issue:** the two-layer audit + acompte are tracked as a dedicated +GitHub issue (extends #498). See #458 (coordination roadmap) for the pointer.