Skip to content

docs(taxonomy): #141 non-card census + AIF cross-ref enrichment proposal - #609

Merged
jsboige merged 1 commit into
masterfrom
docs/141-noncard-census
Jun 29, 2026
Merged

docs(taxonomy): #141 non-card census + AIF cross-ref enrichment proposal#609
jsboige merged 1 commit into
masterfrom
docs/141-noncard-census

Conversation

@jsboige

@jsboige jsboige commented Jun 29, 2026

Copy link
Copy Markdown
Contributor

Summary

Grounds #141 in the current taxonomy state (the issue text is ~2 years old; the DatasetUpdater + translation PRs have moved the state considerably since). The deliverable is a read-only census + proposal0 write Cards/, pre-tag safe.

This is the PRIMAIRE TRACK-3 dispatched by ai-01 (post-coupure re-dispatch, deep-queue).

Key finding — #141 reframed by the census

The original enrichment target is DONE. Non-card node text (descriptions / examples / titles) is 100 % across 8 languages on both datasets. The "GPT-4 enrichment script" the issue asks to adapt is already adapted — it is the modern DatasetUpdater (PR #210, OpenAI SDK v2.10.0, gpt-5.5 primary).

The genuinely-open residue is the AIF cross-reference graph (crossLink_* 8 relationship columns + AIF_skos* 4 SKOS mappings): schema is ready, content ~0 % populated on non-card nodes.

Census numbers (measured on ba8e4a6c)

Dataset Non-card nodes Text ×8 langs crossLink_* (8 cols) AIF_skos* (4 cols)
Fallacies 1232 100 % 15 / 9856 (0.2 %) 26 / 4928 (0.5 %)
Virtues 110 100 % 110 / 880 (12.5 %) 220 / 440 (50 %)

Proposed method (gpt-5.5 assist + expert gate — NOT auto-write)

4 stages: ground (this census) → gpt-5.5 candidate generation (/v1/responses + reasoning.effort=low, context = the node's 100 %-populated desc/example + siblings + AIF scheme + 8 verb defs) → drift-free sidecar dry-run (cell-by-cell, method #595) → expert/jsboige ratification gate (AIF/Walton mappings are specialised — memory: "Anti-Fab Validator: Walton scheme = WARN" — LLM candidates must not be auto-ratified; same discipline as #192) → OWL/2sxc export (#130/#136).

"Adapt the GPT-4 script" = not a rewrite: add a crossLink prompt + task config to DatasetUpdater (gated post-release on jsboige GO + a schema decision: bare decimal_path vs structured {target, note}).

Files (2, docs + read-only script)

Scope

  • 0 write Cards/, 0 AssetConverter code change (pre-tag safe)
  • ✅ Base ba8e4a6c, docs/script only
  • ✅ SDDD bookend: grounded in actual CSV state (code = truth), not the 2-year-old issue text

Relates to #130 (OWL), #136 (2sxc), #498 (AIF scale-up pilots — the few filled cells come from here). Reframes #141: text-enrichment scope complete; the cross-reference graph is the open work, gated post-release on jsboige GO + schema decision.

Grounds #141 in the CURRENT taxonomy state (issue text is ~2 years old;
DatasetUpdater + translation PRs have moved the state considerably).

Census finding (read-only script on ba8e4a6, reproducible like #600/#606):
  - Non-card nodes: Fallacies 1232, Virtues 110 (Scenarii/Rules out of scope).
  - TEXT enrichment (desc/example/title x 8 langs) = 100% on both datasets
    => original #141 scope item 3 (descriptions + examples + translations)
    is DONE. No text-enrichment gap to script.
  - The "GPT-4 enrichment script" = already adapted: modern DatasetUpdater
    (PR #210, OpenAI SDK v2.10.0, gpt-5.5). Not a rewrite.

Reframes #141: the genuinely-open residue is the AIF cross-reference graph
(crossLink_* 8 relationship cols + AIF_skos* 4 SKOS mappings). Schema is
READY (columns exist), content ~0% on non-cards:
  - Fallacies: crossLink 0.2%, AIF_skos 0.5% (barely started)
  - Virtues: crossLink_Opposes + skosDirectRef/MappingType 100% via #498
    pilots, but 7 OTHER crossLink verbs = 0%
  - ~16700 empty cross-ref cells where schema is waiting.

Proposal: 4-stage method (ground -> gpt-5.5 candidate via /v1/responses
effort=low -> drift-free sidecar dry-run -> expert/jsboige ratification
gate -> OWL/2sxc export). NOT auto-write: AIF/Walton mappings are
specialised (memory: "Anti-Fab Validator: Walton scheme = WARN"), and
Cards/ is frozen pre-tag. "Adapt the script" = add a crossLink prompt +
task config to DatasetUpdater, GATED post-release on jsboige GO + schema
decision (single decimal_path vs structured {target,note}).

Overlaps #498 (AIF scale-up pilots produced the few filled cells).
Feeds dependencies #141 declares: #130 (OWL/SKOS) + #136 (2sxc).

Scope: docs + read-only script only. 0 write Cards/, 0 AssetConverter code
change (pre-tag safe). Base ba8e4a6.

Relates to #130, #136, #498.

Co-Authored-By: Claude-Code <noreply@anthropic.com>

@clusterManager-Myia clusterManager-Myia left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[NanoClaw]

LGTM — well-grounded reframe of #141 + read-only reproducible census. The proposal's core move is sound: instead of adapting the 2-year-old "GPT-4 enrichment" ask, it measures the actual current taxonomy state and shows the original text-enrichment target is DONE (non-card node text 100% × 8 langs, both datasets, via modern DatasetUpdater #210), so the genuinely-open residue is the AIF cross-reference graph (crossLink_* 8 cols + AIF_skos* 4 cols) — schema ready, content ~0% filled.

Verified firsthand :

  • Script is genuinely read-only (141-noncard-census.py) : only open() read + csv.DictReader + print. 0 write under Cards/, 0 network, deterministic — the pre-tag-safe claim holds. Reproducible (python docs/taxonomy/141-noncard-census.py).
  • Census math internally consistent : Fallacies 1232 non-card × 8 crossLink cols = 9856 → 15/9856 = 0.15% ≈ 0.2% ✓ ; 110 × 8 = 880 → 110/880 = 12.5% ✓ ; AIF_skos 1232 × 4 = 4928 → 26/4928 = 0.5% ✓ ; Virtues 110 × 4 = 440 → 220/440 = 50% ✓. Every number in the body checks out against its own cell arithmetic.
  • Reframe logically grounded : non-card node = card-flag empty row (family/subfamily headers), correct definition ; the "text DONE, cross-ref open" split is the right prioritization (cross-ref fills drive #130 OWL + #136 2sxc exports).
  • Expert-gate discipline correct : the proposed method keeps a human/expert ratification gate for AIF/Walton mappings (specialized semantics — "Anti-Fab Validator: Walton scheme = WARN"), gpt-5.5 only proposes candidates + sidecar dry-run (method #595). Matches the cluster's anti-fabrication stance, no auto-write.

Honest caveat (limit of automated review) : I verified the script is read-only and the census arithmetic is internally consistent, but I did not re-execute the script against the live CSVs to independently confirm the raw counts (1232 / 110 / 15 / 26...) are the true current state on ba8e4a6c. The claim is reproducible/auditable (anyone can re-run), which is the right bar for a docs/proposal PR — but the base counts themselves are taken on the author's "measured" assertion, not re-measured by me.

0 leak (the lone grep hit was max_output_tokens containing "token" — API param, not a secret). mergeable_state: clean.

— NanoClaw (myia-ai-01)

@jsboige
jsboige merged commit 434edd6 into master Jun 29, 2026
3 checks passed
@jsboige
jsboige deleted the docs/141-noncard-census branch June 29, 2026 10:55
jsboige added a commit that referenced this pull request Jul 1, 2026
…ue (0 Cards/) (#619)

TERTIAIRE of ai-01 deep-queue supersede (msg-...370u0q). Read-only
scoping of the #202 "bulk EN translation" backlog.

Reframe (stale-dispatch pattern, like #609/#618): #202's Phase-2 bulk
counts (Simple_name_en 1348 empty, political_example 1373 empty) are
2-year-old and contradict the live census.

Census on master 18b4d02:
  - Fallacies core text (desc/example/text x8 langs) = 100% (census #609).
  - Only 6 _en columns total. EN > FR source on the bulk-secondary ones:
      nom_vulgarisé (FR=3%) vs Simple_name_en (EN=4%) -> 25 translatable
      exemple politique (FR=3%) vs political_example_en (EN=2%) -> 10
    => EN cells were curated directly in English; FR SOURCE is ~97% empty.
       Cannot translate content never authored.
  - Genuinely FR->EN translatable residue = ~35 cells total (trivial).
  - Scenarii baratineur leak = 0/167 (Phase-2 resolved). Virtues i18n =
    100% (#218/#236/#246/#290/#295). Rules PT fixed (#306).
  - Real i18n gap = link_* (lane #600/#618, DONE), not text.

Recommendation: close or re-scope #202. The concrete residue is (a) FR
source editorial authoring (lane #191, human, not translation), (b) a
~35-cell micro-pass, (c) link_*/AIF lanes already tracked. Do NOT launch
a bulk EN run — no source to translate on secondary cols, core is 100%.
Memory i18n-coverage-gap-is-link-urls honored (measure FR-relatively).

Scope: docs/taxonomy/202-bulk-en-scoping.md only. 0 write Cards/, 0
AssetConverter code change (pre-tag safe). Base 18b4d02.

Relates to #202, #191, #609, #600, #618.

Co-authored-by: Your <your.email@example.com>
Co-authored-by: Claude-Code <noreply@anthropic.com>
jsboige added a commit that referenced this pull request Jul 1, 2026
…ase 2 jsboige-gated (#621)

Idle of ai-01 deep-queue supersede (msg-...370u0q): #415 .git weight
analysis. Read-only audit + reproducible tool. 0 write Cards/.

Stale-issue refresh (4th stale-dispatch this session, after #609/#618/#619):
#415 body (measured on 0bf7785, 2026-06-01) states "~1,4 GB tracked at
HEAD" + proposes Phase 1 (gitignore + git rm --cached). Phase 1 has SINCE
been executed and merged:
  - PR #416 (94b4371) "stop tracking 968 MB regenerable artifacts"
  - PR #501 (ff03147) "#415 untrack 10.5 MB 2sxc install module"

Audit on master 18b4d02 (tools/git-weight-audit.py, reproducible):
  - size-pack = 2.05 GiB (matches issue; Phase 1 doesn't reduce pack).
  - Regenerable zones = 0 at HEAD (untracked), survive only in history:
      Published/ .NET builds 1.2 GB (42 paths)
      DNN Downloads zips     206 MB (12)
      2sxc/DNN .resources    152 MB (197)
  -> Phase 1 is CLOSED. The 2 GB is 100% historical.

Phase 2 (filter-repo) = the ONLY remaining weight-reducer (~1.5-1.6 GB
reclaimable -> <200 MB clone). DESTRUCTIVE, jsboige-gated (rewrites all
SHAs, force-push, all machines re-clone). Not autonomous.

Sources to PRESERVE (Phase 3, LFS/external, jsboige decision):
  Cards/Packaging *.ai/pdf/svg 97 MB, Sketch .sketch 45 MB, card PNG 76 MB.

Note: Mindmap SVGs (48 MB) still tracked at HEAD though regenerable —
not covered by #416; byte-stable deliverables (#565); left conservative.

Interim onboarding mitigation (non-destructive, actionable NOW): document
`git clone --filter=blob:none` (no partialclone default currently set).
This is the one concrete release-safe action recommended.

Scope: tools/git-weight-audit.py + docs/repo/415-git-weight-audit.md only.
0 write Cards/, 0 AssetConverter code change. Base 18b4d02.

Relates to #415, #416, #501, #134.

Co-authored-by: Your <your.email@example.com>
Co-authored-by: Claude-Code <noreply@anthropic.com>
jsboige added a commit that referenced this pull request Jul 1, 2026
…sume, anti-fab closed-set (#623)

PRIMAIRE of ai-01 deep-queue supersede #3 (msg-...v95b6l): scales the #620
pilot (28 nodes, 0 fabrication) to ALL Fallacies non-card nodes (1232),
same closed-set anti-fab design. Dry-run sidecar generator. 0 write Cards/.

Anti-Fab design (validated 0 fabrication on the pilot, holding at scale):
  - cross-link targets picked from enumerated REAL nodes (siblings +
    parent + children + 7 family roots) — cannot invent a decimal_path.
  - AIF scheme tokens picked from the 60-token observed Walton vocab —
    cannot invent a scheme name.
  - post-generation validation re-checks target in index, token in vocab,
    verb in 8, mappingType in observed set.

CHECKPOINT/RESUME (spans multiple ticks at ~12s/node -> ~4h for 1232):
  - one JSON line per node in tmp/141-aif-fullscale.jsonl (not committed).
  - re-run skips already-processed nodes (load_ckpt on source_dp).
  - flush after each node — survives crash/tick boundaries.
  - --limit N caps new nodes per run; --finalize aggregates -> sidecar.

Modes:
  python docs/taxonomy/141-aif-fullscale.py              # resume, all
  python docs/taxonomy/141-aif-fullscale.py --limit 50   # cap new nodes
  python docs/taxonomy/141-aif-fullscale.py --finalize   # tmp -> sidecar (no API)

Smoke-tested (--limit 3): 3/3, 0 errors, 12s/node, non-card=1232 (matches
#609 census), vocab=60. Full run launched in background this tick;
sidecar CSV/JSON + coverage/WARN report land in a follow-up PR via
--finalize. Only WARN at scale = skos:relatedMatch/exactMatch (legit SKOS
predicates outside the observed {broad,close,narrow}Match set -> a schema
decision for the expert gate, NOT fabrications).

gpt-5.5 /v1/responses + reasoning.effort=low + json_object (memory
gpt55-responses-api-effort-low). Key read by path from session scratchpad,
never inlined. DRY-RUN: checkpoint+interim to tmp/ (not committed); final
sidecar to docs/taxonomy/ only when aggregated.

Scope: docs/taxonomy/141-aif-fullscale.py only. 0 write Cards/, 0
AssetConverter code change. Base d0856aa.

Relates to #141, #609, #620, #130, #192.

Co-authored-by: Your <your.email@example.com>
Co-authored-by: Claude-Code <noreply@anthropic.com>
jsboige added a commit that referenced this pull request Jul 1, 2026
…abrication (#626)

* docs(taxonomy): #141 AIF Stage-1 full-scale results — 1232 nodes, 0 fabrication

PRIMAIRE of ai-01 deep-queue supersede #3 (msg-...v95b6l): the full-scale
AIF Stage-1 candidate sidecar + coverage/WARN report. The generator was
shipped in #623; this is the run output (1232/1232 non-card nodes,
gpt-5.5 assist, dry-run, 0 write Cards/).

Headline — anti-fab closed-set HOLDS at full scale:
  - 1232 nodes covered (100% of Fallacies non-cards, the #609 gap).
  - 3850 crossLinks (3.12/node, 100% nodes) + 2310 AIF refs (1.88/node,
    96% nodes).
  - confidence mean 0.73, median 0.72 (1301 high >=0.8 / 2460 mid / 89 low).
  - FABRICATION = 0. invalid targets = 0. invalid verbs = 0.
    The closed-set design (targets from enumerated real nodes, AIF tokens
    from the 60-token observed Walton vocab) contains LLM fabrication
    completely across 1232 nodes.

crossLink verb mix: IsRelatedTo 42% / Leverages 33% / Mirrors 15% /
Allows 7% / Inverts 2% / Opposes 1% / PredatesOn 1% / Denounces <1%.
Honest improvement vs pilot: Opposes/PredatesOn/Denounces were 0 in the
28-node pilot (flagged limitation) -> non-zero at scale (sample was too
small to surface them); still sparse, conservative-but-incomplete.

Top Walton DirectRef schemes well-represented: Bias_Inference 262,
Sign_Inference 119, Ethotic_Inference 117, EvidenceToHypothesis 94.

WARN = 87 nodes (7%), ALL bad_map:* (mappingType outside the observed
{broad,close,narrow}Match set: relatedMatch 78, none 5, exactMatch 2,
noMatch 2). These are LEGIT SKOS predicates -> a schema-extension
decision for the expert gate, NOT fabrications. none/noMatch = hedge
candidates to DROP during ratification.

Sidecars: 141-aif-candidates-fullscale.csv (1.1 MB, flat, reviewable,
committed) + report. The structured JSON (1.4 MB) is regenerable via
--finalize from the checkpoint and is NOT committed (repo already
2.05 GiB per #415/#621 — limit additive weight).

1 node (6.3.1.2.3.1.4 Hypocrisie morale) failed once on an API read-
timeout, recovered on a 1-node resume (0 content/parse errors).

Scope: docs/taxonomy/141-aif-candidates-fullscale.csv + report only.
0 write Cards/, 0 AssetConverter code change. Base 0e39425.

Next (gated post-release): Stage 2 dry-run diff, Stage 3 expert/jsboige
ratification gate (decide relatedMatch/exactMatch schema extension,
drop none/noMatch hedges, prioritize >0.8 candidates), Stage 4 ->
#130 OWL / #136 2sxc.

Relates to #141, #609, #620, #130, #136, #192.

Co-Authored-By: Claude-Code <noreply@anthropic.com>

* docs(taxonomy): #141 AIF Stage-2 dry-run diff — 9 conflict / 2 confirm / 5 silent

Read-only diff of the Stage-1 sidecar (#626) against the 12 non-card nodes
that already carry an AIF value on the taxonomy (0.97% — the other 1220 are
entirely net-new, no conflict possible). High-signal prep for the expert gate:
- 2 CONFIRM: gpt-5.5 independently re-derived the exact expert token
  (Ignorance_Inference, VagueVerbalClassification_Inference) — closed-set
  produces real Walton mappings.
- 9 CONFLICT: 1 field-swap/churn node (Pente glissante) + 5 frank token
  disagreements → gate must adjudicate, not auto-apply.
- 5 SILENT: generator stayed silent on a filled field → gate must preserve
  existing values.

Adds 141-aif-stage2-diff.py (read-only, 0 Cards/ write), 16-row diff.csv, and
a Stage-2 section to the report. Same scope/honors pre-tag freeze.

Co-Authored-By: Claude-Code <noreply@anthropic.com>

---------

Co-authored-by: Your <your.email@example.com>
Co-authored-by: Claude-Code <noreply@anthropic.com>
jsboige added a commit that referenced this pull request Jul 1, 2026
…commendation (#627)

* docs(taxonomy): #141 Stage-3 expert adjudication package + closure recommendation

Stage-3 (PRIMAIRE dispatch ai-01): one-pass adjudication package for the 12
non-card nodes that already carry an AIF value (0.97% of 1232). Sourced from
the reproducible stage2-diff.csv (anti-fab): 2 CONFIRM (ratify), 9 CONFLICT
across 6 nodes (advisory recos with Walton reasoning, expert decides), 5 SILENT
(preserve existing). No auto-apply — every row is a human gate call. Surfaces a
cross-cutting scheme-vs-conflict tier convention (_Inference/Scheme -> DirectRef,
_Conflict -> ExceptionRef) for the gate to adopt or reject.

Closure (TERTIAIRE): synthesis of Phases 1-3 (census #609 + full-scale #626 +
Stage-2/3) -> recommend #141 closed as delivered-and-gated, with the 3 close
criteria for the expert gate.

Read-only docs, 0 Cards/ write, 0 AssetConverter change. Honors pre-tag freeze.

Co-Authored-By: Claude-Code <noreply@anthropic.com>

* docs(taxonomy): #141 WARN bad_map:* triage — 87 nodes (80 adopt / 7 drop)

Completes the #141 expert-gate package (the schema-extension decision, alongside
Stage-3 adjudication + closure already in this PR). Triages the 87 non-card
nodes whose mappingType fell outside the observed {broad,close,narrow}Match set:

- skos:relatedMatch 78 + skos:exactMatch 2 -> ADOPT (extend observed set; legit
  SKOS, enriches semantics at zero fabrication risk; exactMatch spot-verify).
- none 5 + skos:noMatch 2 -> DROP (weak hedges; leave mappingType empty).

80 ADOPT / 7 DROP. Confirms these are NOT fabrications (0 across 1232 nodes) —
a schema call for the gate. Sourced from the structured JSON (source of truth
for WARNs — the flat CSV undercounts by ~23 nodes that have a mappingType but
0 DirectRef/ExceptionRef rows). Reproducible via 141-aif-warn-triage.py ->
87-row reviewable CSV.

Read-only docs+tool, 0 Cards/ write, 0 AssetConverter change. Honors pre-tag freeze.

Co-Authored-By: Claude-Code <noreply@anthropic.com>

---------

Co-authored-by: Your <your.email@example.com>
Co-authored-by: Claude-Code <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants