test(ontology): #133 e2e on real generated ontology — surfaces 2nd silent-false-pass bug (rdf:type dropped by OWL2XML round-trip) - #486
Merged
Conversation
…lent-false-pass bug Loads the committed generated ontology (docs/ontology/argumentum.owl, the real OwlDocumentConfig.CreateOwlDocument output) and runs the production validation path against it. This completes the #133 e2e proof that #482 (synthetic in-memory adapters) started — and surfaces a SECOND silent-false-pass bug the #480→#481→#482 lane missed because it only exercised the in-memory path. ROOT CAUSE (measured on the reloaded file): OWLSharp's OWL2XML serializer DROPS the rdf:type and skos:inScheme annotation assertions during serialization — neither survives the round-trip (rdf:type == 0, inScheme == 0 after reload). The OwlAdapter readers find concepts/schemes by scanning AnnotationAxioms for rdf:type, so on any LOADED file they return empty. OwlOntologyValidationTests.ValidateMultilingualAnnotations and .ValidateAIFMappings then hit their `if (concepts.Count == 0) return true;` guard and report PASS without inspecting anything. The real content IS present (2816 prefLabels, 10 AIF matches, 1510 class declarations) — the validator simply cannot see it. So #133's "confidence restored" premise does NOT hold for the production load-and-validate path: the silent false-pass is still alive there, for a different root cause than the #480 RDFResource type-mismatch. 4 characterization tests (all green, pinning current broken behavior): 1. rdf:type==0 and inScheme==0 after round-trip (prefLabel survives as contrast). 2. GetResourcesByType(Concept/ConceptScheme) returns empty on the loaded file. 3. The reloaded ontology DOES contain real content (prefLabel>1000, matches>0, classDecls>1000) — reader defect, not data defect. 4. Production validator returns TRUE for annotation + AIF checks on the loaded ontology — the silent false-pass, decisively pinned. When the fix lands (readers locate concepts via surviving annotations — prefLabel/definition/example subjects, or filtered DeclarationAxioms — NOT rdf:type/inScheme), these assertions flip to the honest "detection works" form. The fix is a prod behavior change with release-gate implications → coordinator scope; this PR surfaces it rather than shipping a unilateral prod change. Deterministic, key-free, release-independent. NEW additive file — nothing modified. Co-Authored-By: Claude <noreply@anthropic.com>
clusterManager-Myia
left a comment
clusterManager-Myia
left a comment
Collaborator
There was a problem hiding this comment.
[NanoClaw]
Solid characterization test PR that surfaces a real bug: the OWL2XML round-trip silently drops rdf:type and skos:inScheme annotations from the generated ontology.
Key observations:
- Loads the real
docs/ontology/argumentum.owl(~5MB), not a toy fixture — good fidelity - Proves the drop mechanism end-to-end: write → OWL2XML serialize → read back → annotations gone
- Readers return empty collections where they should not, validators pass silently — this is the dangerous part
- Tests are correctly GREEN, pinning the broken behavior as a regression guard before the fix lands
One concern: EntityReader.ParseEntityList() has a documented _skippable field check that silently filters unknown annotations. The test confirms this masking behavior but does not instrument _skippable directly — worth a follow-up to confirm the fix will assert on skip-count, not just check final counts.
LGTM — this is the right way to surface a silent-false-pass before fixing it.
jsboige
added a commit
that referenced
this pull request
Jun 16, 2026
…nt-false-pass dead (#489) The #481 reader fix only covered the IN-MEMORY path. On a LOADED ontology (rdf:type + skos:inScheme dropped from the reloaded AnnotationAxioms by OWLSharp's OWL2XML round-trip), GetResourcesByType(Concept)/GetConcepts returned empty -> ValidateMultilingualAnnotations/ValidateAIFMappings hit their `concepts.Count == 0 -> return true` guard -> PASS without inspecting (the 2nd silent-false-pass, characterized by #486). Verified reloaded breakdown (probed on the real generated ontology): prefLabel=2816, definition=2816, example=2816, narrower/broader=1407, broadMatch=57, closeMatch=10, narrowMatch=3, hasTopConcept=1; rdf:type=0, inScheme=0 among AnnotationAxioms. Fix is READ-PATH ONLY (serializer untouched, per dispatch scope). When the rdf:type scan is empty, locate entities via the surviving SKOS annotations: - skos:Concept -> distinct subjects of skos:prefLabel (~1305 resolved) - skos:ConceptScheme -> subject of skos:hasTopConcept In-memory path preserved (rdf:type present -> early-return, no fallback), so the #482 in-memory live-path proofs still hold. Concepts deduped by URI string (not RDFResource.Equals) to avoid the equality bug class of #480. OwlE2EGenerationValidationTests (#486) flipped from bug-characterization to a genuine-pass regression suite: (1) rdf:type/inScheme drop still real (now benign), prefLabel survives (2) readers now resolve concepts (>1000) + scheme (NotBeEmpty) (3) real content now resolvable (>1000 distinct concept subjects) (4) prod validator genuinely inspects + passes (skip guard unreachable) Tests: Ontology namespace 25/25 green; full suite 259 passed / 0 failed / 5 skipped (no regression). Dispatch msg-...irkf5i primaire, base 4ac52e2. Read-path only: no write/serialize, no CSV/template/config touched. Co-authored-by: Your <your.email@example.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Completes the #133 e2e OWL validation proof by running the production validation path against the REAL generated ontology (
docs/ontology/argumentum.owl). This is the real-data counterpart to #482's synthetic in-memory proof — and it surfaces a second silent-false-pass bug that the #480→#481→#482 lane missed (that lane only exercised the in-memory path).The finding — #133's premise does not hold for the prod load path
The production OWL validator STILL silent-false-passes on any ontology loaded from a file. The #481 fix killed the silent false-pass in-memory (proven by #482), but the real production path (
OwlOntologyValidationTests.LoadOntology→OwlAdapter.FromFile→ validate) is still broken, for a different root cause than the #480RDFResource.Equalstype-mismatch.Root cause (measured on the reloaded file)
OWLSharp's OWL2XML serializer drops the
rdf:typeandskos:inSchemeannotation assertions during serialization — neither survives the round-trip:rdf:typeskos:inSchemeskos:prefLabelskos:exactMatch/closeMatch/relatedMatchskos:hasTopConceptThe OwlAdapter readers locate concepts/schemes by scanning
AnnotationAxiomsforrdf:type. On a loaded filerdf:typeis absent →GetResourcesByType(Concept)returns empty →ValidateMultilingualAnnotations/ValidateAIFMappingshit theirif (concepts.Count == 0) return true;guard → report PASS without inspecting anything. The ontology demonstrably contains 2816 prefLabels + 10 AIF matches — the validator just cannot see them.Tests (4, all green — characterization of current broken behavior)
LoadedOntology_RdfTypeAndInScheme_DroppedByOwl2XmlRoundTrip— pinsrdf:type==0,inScheme==0, withprefLabel>0as contrast.LoadedOntology_ConceptAndSchemeReaders_ReturnEmpty_BugPinned—GetResourcesByType(Concept/ConceptScheme)empty on loaded file.LoadedOntology_ContainsRealContent_ReadersCannotSee— smoking gun: 2816 prefLabels + 10 matches + 1510 class declarations present; reader defect, not data defect.ProdValidator_SilentFalsePass_StillActiveOnLoadedOntology— production validator returns TRUE for annotation + AIF checks on the loaded file (the silent false-pass, decisively pinned).When the fix lands, assertions flip to the honest "detection works" form.
Recommended fix (coordinator scope)
The readers must locate concepts via annotations that survive the round-trip — not
rdf:type/inScheme. Options:skos:prefLabel/skos:definition/skos:example(literal-valued, survive), and/orDeclarationAxioms(OWLClassminus the AIF/restriction/scheme classes), and/orskos:hasTopConcept.This is a prod behavior change with release-gate implications (validation that currently false-passes would start running for real), so it's left to coordinator scoping rather than a unilateral worker change. Filing as the #133 deliverable + bug report.
Validation
dotnet testfull suite: 252 pass / 0 fail / 5 skip (baseline 246 + test(parsing): #204 Fallacy ClassMap regression — Optional localized columns guard (+2) #485 +2 + these 4).cc @jsboige — this inverts #133's "confidence restored"; flagging before any release/validation decision.
🤖 Worker po-2024 (dispatch #133 primaire)