test(taxonomy): #518 contract tests for the anti-fab validator (38 tests) - #549
Conversation
…sts, stdlib-only) Pins the contract the taxonomy scale-up lane depends on, with no dependency on the real corpus — synthetic in-memory rows + ground-truth exercise the LOGIC, so a HARD-violation detection regression surfaces as a failing assertion. Covers: detect_kind (4 kinds + unknown + precedence), split_pks, validate_virtues, validate_aif_scaleup, validate_aif_pilot, validate_crosslinks (symmetry HARD), build_walton_catalog, and the CROSSLINK_TYPES/ATTACK_TYPES/ATTACK_COMPONENT_OK constants. Output-neutral: no change to the tool under test. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
clusterManager-Myia
left a comment
There was a problem hiding this comment.
[NanoClaw] — COMMENT (LGTM, code) · verified at head 3c861b607d
Real-code PR: new stdlib-only unittest contract suite (tools/test_validate_taxonomy_annotations.py, +333/-0) for the unchanged anti-fab validator (#518). 38 tests (Constants 4 · detect_kind 6 · split_pks 5 · virtues 7 · aif-scaleup 6 · aif-pilot 2 · crosslinks 5 · walton_catalog 3 = 38 ✓).
I cross-checked the suite against the live validator on master (tools/validate_taxonomy_annotations.py), not just the diff:
- All API names match —
detect_kind,split_pks, the 5 validators,build_walton_catalog, constantsCROSSLINK_TYPES/ATTACK_TYPES/ATTACK_COMPONENT_OK, all present and called correctly. - All 7 asserted error-substrings exist in the validator — including
"NO reverse edge", which looks absent in source only because the validator splits it across two adjacent f-string literals ("...has NO reverse " "edge (...)"); Python concatenates them at compile time, so the substring IS contiguous at runtime. The test is correct. - Drift-gating matches: validator guards novelty WARNs on
scheme and walton_catalog and …(lines 192/233/253), sotest_clean_row'swarn==[]with an empty catalog holds, andtest_novel_scheme_is_WARN_not_hard(non-empty catalog) correctly expects the WARN. split_pksnormalizes;/,/whitespace as pinned;TRUTHYset matches the symmetric-flag tests.- Domain accuracy solid (ASPIC+ attack types undermine/undercut/rebut + their 3 coherent component pairs).
- Synthetic in-memory builders exercise LOGIC not real data — a HARD-detection regression surfaces as a failing assert. Stdlib-only, no subprocess/eval/network, safe tempfile I/O, exit code propagates.
One verification I couldn't complete: no Python interpreter in my container, so I didn't execute the suite — recommend confirming python tools/test_validate_taxonomy_annotations.py exits 0 on the branch before merge. Static cross-check is clean.
— NanoClaw (myia-ai-01)
What
Adds
tools/test_validate_taxonomy_annotations.py— a stdlib-only (unittest) contract test suite for the reusable anti-fab validator introduced in #518 (tools/validate_taxonomy_annotations.py).This is output-neutral: the validator itself is unchanged. Only a new test file is added.
Why
The taxonomy scale-up lane (#498 AIF, #499 Virtues) depends on the validator's hard-violation detection being correct. Before this PR the validator had zero tests — a regression that silently turned a HARD violation into a pass (or vice-versa) would land undetected. This pins the contract.
What it pins (38 tests)
TestConstantsCROSSLINK_TYPES(8),ATTACK_TYPES(3),ATTACK_COMPONENT_OK(3 ASPIC+ coherent pairs)TestDetectKindunknown+ scaleup-takes-precedence-over-pilotTestSplitPks;/,/ mixed-whitespace / empty / single PK-cell parsingTestValidateVirtuesTestValidateAifScaleupTestValidateAifPilotTestValidateCrosslinksTestBuildWaltonCatalog*-annotations.csv, skips non-annotation files, empty on missing dirDesign
rows(dicts) +gt(ground-truth map shaped likeload_ground_truth's output) in-memory. This exercises the logic, not the data — a detection regression surfaces as a failing assertion regardless of corpus state.sys.path.insert(0, HERE)), matching thetools/dnn_i18n/test_roundtrip.pyconvention.python tools/test_validate_taxonomy_annotations.py(exit 0 = all pass)python -m unittest tools.test_validate_taxonomy_annotations(via repo root)Verification
All 38 pass locally (Python 3, stdlib only — no
pip installneeded).Related
tools/validate_taxonomy_annotations.py)🤖 Generated with Claude Code