feat: synchronize official O*NET construct catalog

CHANGELOG.md

@@ -17,6 +17,10 @@ All notable changes to this project are documented here. Format follows
   now persist in normalized tables. Database and application validation require
   same-Post verbatim evidence, and authorized Post detail exposes provenance
   without internal identifiers or numerical scores (ADR 0249).
+- An operator-only O*NET 31.0 catalog synchronizer now imports every official
+  cognitive-ability, work-style, and work-activity Content Model element with
+  stable IRIs, descriptions, attribution, and a deterministic source digest;
+  conflicting release metadata fails closed (ADR 0250).
 
 - The DOT/FJA Data/People/Things worker-function taxonomy is now published
   in the canonical ontology: all 24 worker functions carry the official

backend/tests/test_api.py

@@ -239,6 +239,11 @@
     / "migrations"
     / "0238_occupational_construct_assertion.sql"
 )
+_OCCUPATIONAL_CATALOG_MIGRATION = (
+    Path(__file__).resolve().parents[2]
+    / "migrations"
+    / "0239_occupational_construct_catalog.sql"
+)
 
 
 def _postgres_available() -> bool:
@@ -401,6 +406,7 @@ def seeded_db(demo_analyst_token):
             cur.execute(_LEFTOVER_MAP_AXIS_MIGRATION.read_text())
             cur.execute(_CHANNEL_EVIDENCE_MIGRATION.read_text())
             cur.execute(_OCCUPATIONAL_CONSTRUCT_MIGRATION.read_text())
+            cur.execute(_OCCUPATIONAL_CATALOG_MIGRATION.read_text())
             cur.execute(_LEFTOVER_MAP_UNEXPLAINED_MIGRATION.read_text())
             cur.execute(_LEFTOVER_MAP_CROSS_SHARE_MIGRATION.read_text())
             cur.execute(_LEFTOVER_MAP_RECONSTRUCTION_MIGRATION.read_text())

docs/adr/0248-occupational-construct-evidence-boundary.md

@@ -68,8 +68,9 @@ affective science has no single default category set.
   evidence relationship without claiming that the first source mention is a
   measured person attribute.
 - Complete O*NET breadth remains available through its maintained RDF graph;
-  runtime ingestion and persistence are still unavailable until a connector
-  and normalized assertion tables are accepted and shipped.
+  ADRs 0249 and 0250 add normalized assertion persistence and official catalog
+  synchronization. Record extraction remains unavailable until its separate
+  contextual-orchestrator contract is accepted and shipped.
 - Actual affect stays absent unless the evidence names a conforming affect
   vocabulary and supports the reaction; work style is never relabeled affect.
 - Unsupported equivalence and causal links fail closed rather than becoming

docs/adr/0250-official-occupational-construct-catalog-sync.md

@@ -0,0 +1,69 @@
+# ADR 0250: Official occupational construct catalog synchronization
+
+**Status:** Accepted
+**Date:** 2026-08-27
+**Extends:** [ADR 0248](0248-occupational-construct-evidence-boundary.md), [ADR 0249](0249-occupational-construct-assertion-persistence.md)
+
+## Context
+
+The assertion store accepted by ADR 0249 needs a complete, reviewable catalog
+before contextual-orchestrator can select a construct. Asking a model to invent
+an O*NET label or IRI would defeat ADR 0248. Shipping a hand-maintained subset
+would also drift from O*NET's quarterly releases and omit most of the requested
+cognitive and behavioral domain.
+
+O*NET 31.0 publishes a machine-readable Content Model Reference with stable
+element identifiers, names, hierarchy positions, descriptions, release
+documentation, and CC BY 4.0 attribution terms. Its hierarchy explicitly places
+cognitive abilities below `1.A.1`, work styles below `1.D`, and work activities
+below `4.A`. These are source classifications, not a LineageWeave heuristic.
+
+## Decision
+
+1. An operator-only synchronizer reads the fixed HTTPS O*NET 31.0 Content Model
+   Reference JSON document. The URL, release, vocabulary IRI, license IRI, and
+   attribution are code-reviewed constants; runtime input cannot redirect the
+   process to an arbitrary host.
+2. The synchronizer imports every element at or below the three published
+   hierarchy roots: `1.A.1` as `cognitive_ability`, `1.D` as `work_style`, and
+   `4.A` as `work_activity`. It preserves official labels, optional descriptions,
+   and permanent `https://data.onetcenter.org/element/{element_id}` IRIs.
+3. The canonical decoded JSON SHA-256 is stored on the vocabulary release.
+   The reviewed O*NET 31.0 document digest is
+   `cb25e83a25c355dba035afdfc6b23ed8706a939d5f5021ed772d554ea49afb06`;
+   synchronization rejects any other digest before opening a database
+   transaction. Replaying the same document is idempotent. A changed document
+   under the same release, or conflicting construct metadata, aborts instead
+   of rewriting history. The reviewed document contains 3,006 source rows and
+   admits 2,529 governed constructs: 29 cognitive abilities, 26 work styles,
+   and 2,474 work activities.
+4. This catalog does not import occupation ratings, scores, scale values,
+   ability-to-activity linkages, work-style linkages, FJA crosswalks, affective
+   vocabularies, or person/job bindings. Those require their own provenance and
+   decision records.
+5. Catalog synchronization is a prerequisite for extraction. The extractor
+   may select only catalog rows supplied to contextual-orchestrator; it may not
+   mint a label, family, description, or IRI.
+
+## Consequences
+
+- The semantic layer gains the full official breadth needed for catalog-bound
+  cognitive, work-style, and work-activity assertions without copying these
+  terms into the LineageWeave ontology namespace.
+- O*NET descriptions that are absent remain `NULL`; LineageWeave does not fill
+  them with generated prose.
+- Affective reactions and performance interpretation remain unavailable until
+  an authoritative vocabulary and evidence contract are accepted.
+
+## Verification
+
+- Parser tests cover all three roots, exact IRI construction, ignored unrelated
+  rows, malformed payloads, and deterministic source hashing.
+- Schema tests require replay-safe catalog description and source-hash columns.
+- Synchronization tests prove idempotent UPSERTs and post-write exact metadata
+  comparison.
+
+## References
+
+See
+[`docs/doctoring/OCCUPATIONAL_CONSTRUCT_REFERENCES.md`](../doctoring/OCCUPATIONAL_CONSTRUCT_REFERENCES.md).

docs/doctoring/OCCUPATIONAL_CONSTRUCT_REFERENCES.md

@@ -11,6 +11,10 @@ source actually supports and which tempting inferences remain prohibited.
   It is CC BY 4.0; derived products must credit USDOL/ETA, link the license,
   and identify modifications. LineageWeave links rather than remints these
   resources.
+- The O*NET 31.0 Content Model Reference publishes 3,006 hierarchy elements.
+  ADR 0250 admits only the source-defined cognitive-ability (`1.A.1`), work-
+  style (`1.D`), and work-activity (`4.A`) roots and descendants; it preserves
+  blank descriptions as unavailable and stores no occupation rating.
 - The O*NET Content Model separates worker characteristics and requirements
   from occupational requirements. It does not make FJA worker functions
   equivalent to abilities, dispositions, or affect.
@@ -54,6 +58,10 @@ National Center for O*NET Development. (2026). *O*NET 31.0 database* [Data
 set]. U.S. Department of Labor, Employment and Training Administration.
 https://www.onetcenter.org/database.html
 
+National Center for O*NET Development. (2026). *O*NET 31.0 Content Model
+Reference* [Data set]. U.S. Department of Labor, Employment and Training
+Administration. https://www.onetcenter.org/dl_files/database/db_31_0_json/content_model_reference.json
+
 Peterson, N. G., Mumford, M. D., Borman, W. C., Jeanneret, P. R., Fleishman,
 E. A., Levin, K. Y., Campion, M. A., Mayfield, M. S., Morgeson, F. P.,
 Pearlman, K., Gowing, M. K., Lancaster, A. R., Silver, M. B., & Dye, D. M.

docs/product-requirements.md

@@ -86,8 +86,10 @@ continues to pass unchanged.
 Acceptance: SHACL rejects incomplete assertions; ontology tests prohibit FJA
 equivalence and require exact Post/evidence/PROV statement structure. ADR 0249
 adds normalized, semantic-unit-bound persistence and an authorized Post-detail
-projection. Search, graph navigation, extraction, and UI remain unavailable
-until their separate ADR acceptance.
+projection. ADR 0250 synchronizes all official O*NET cognitive-ability,
+work-style, and work-activity Content Model elements into that versioned
+registry without importing ratings. Search, graph navigation, extraction, and
+UI remain unavailable until their separate ADR acceptance.
 
 ### PRD-FR-3 — Bounded ontology exploration
 

lineageweave/occupational_construct_catalog.py

@@ -0,0 +1,192 @@
+"""Synchronize an evidence-safe subset of the official O*NET catalog."""
+
+from __future__ import annotations
+
+import hashlib
+import json
+import re
+from dataclasses import dataclass
+from typing import Any
+
+
+ONET_RELEASE = "31.0"
+ONET_CONTENT_MODEL_URL = (
+    "https://www.onetcenter.org/dl_files/database/"
+    "db_31_0_json/content_model_reference.json"
+)
+ONET_CONTENT_MODEL_CANONICAL_SHA256 = (
+    "cb25e83a25c355dba035afdfc6b23ed8706a939d5f5021ed772d554ea49afb06"
+)
+ONET_VOCABULARY_IRI = "https://www.onetcenter.org/database.html"
+ONET_LICENSE_IRI = "https://creativecommons.org/licenses/by/4.0/"
+ONET_ATTRIBUTION = (
+    "This product includes information from the O*NET 31.0 Database by "
+    "the U.S. Department of Labor, Employment and Training Administration "
+    "(USDOL/ETA). Used under the CC BY 4.0 license. O*NET® is a trademark "
+    "of USDOL/ETA."
+)
+_ELEMENT_ID = re.compile(r"^[0-9]+(?:\.[A-Za-z0-9]+)*$")
+_FAMILY_ROOTS = (
+    ("1.A.1", "cognitive_ability"),
+    ("1.D", "work_style"),
+    ("4.A", "work_activity"),
+)
+
+
+@dataclass(frozen=True)
+class CatalogConstruct:
+    """One exact O*NET Content Model element admitted by ADR 0250."""
+
+    construct_iri: str
+    family_code: str
+    preferred_label: str
+    description: str | None
+
+
+def catalog_content_sha256(payload: dict[str, Any]) -> str:
+    """Hash the deterministic canonical JSON representation of one release."""
+    canonical = json.dumps(
+        payload, ensure_ascii=False, sort_keys=True, separators=(",", ":")
+    ).encode("utf-8")
+    return hashlib.sha256(canonical).hexdigest()
+
+
+def parse_onet_construct_catalog(payload: dict[str, Any]) -> tuple[CatalogConstruct, ...]:
+    """Parse only published cognitive, work-style, and work-activity roots."""
+    if payload.get("table_id") != "content_model_reference":
+        raise ValueError("O*NET payload is not the Content Model Reference")
+    rows = payload.get("row")
+    if not isinstance(rows, list):
+        raise ValueError("O*NET Content Model Reference rows must be an array")
+
+    constructs: dict[str, CatalogConstruct] = {}
+    for row in rows:
+        if not isinstance(row, dict):
+            raise ValueError("O*NET Content Model Reference row must be an object")
+        element_id = row.get("element_id")
+        label = row.get("element_name")
+        if not isinstance(element_id, str) or not _ELEMENT_ID.fullmatch(element_id):
+            raise ValueError("O*NET element_id is malformed")
+        if not isinstance(label, str) or not label.strip():
+            raise ValueError("O*NET element_name must be non-empty")
+        if label != label.strip():
+            raise ValueError("O*NET element_name must not contain outer whitespace")
+        family = next(
+            (
+                family_code
+                for root, family_code in _FAMILY_ROOTS
+                if element_id == root or element_id.startswith(f"{root}.")
+            ),
+            None,
+        )
+        if family is None:
+            continue
+        description_value = row.get("description")
+        if description_value is not None and not isinstance(description_value, str):
+            raise ValueError("O*NET description must be text or null")
+        description = (description_value or "").strip() or None
+        iri = f"https://data.onetcenter.org/element/{element_id}"
+        if iri in constructs:
+            raise ValueError(f"duplicate O*NET construct IRI: {iri}")
+        constructs[iri] = CatalogConstruct(iri, family, label, description)
+    if not constructs:
+        raise ValueError("O*NET catalog contains no governed construct roots")
+    return tuple(constructs[iri] for iri in sorted(constructs))
+
+
+async def sync_onet_construct_catalog(
+    conn: Any,
+    payload: dict[str, Any],
+    *,
+    expected_source_sha256: str = ONET_CONTENT_MODEL_CANONICAL_SHA256,
+) -> int:
+    """Atomically synchronize one immutable O*NET release and verify it exactly."""
+    source_sha256 = catalog_content_sha256(payload)
+    if source_sha256 != expected_source_sha256:
+        raise ValueError("O*NET 31.0 source digest differs from the reviewed release")
+    constructs = parse_onet_construct_catalog(payload)
+    async with conn.transaction():
+        vocabulary_id = await conn.fetchval(
+            """
+            insert into occupational_construct_vocabulary
+                (vocabulary_iri, version_label, license_iri, attribution_text,
+                 source_content_sha256)
+            values ($1, $2, $3, $4, $5)
+            on conflict (vocabulary_iri, version_label) do update set
+                source_content_sha256 = coalesce(
+                    occupational_construct_vocabulary.source_content_sha256,
+                    excluded.source_content_sha256
+                )
+            where occupational_construct_vocabulary.license_iri = excluded.license_iri
+              and occupational_construct_vocabulary.attribution_text = excluded.attribution_text
+              and (
+                  occupational_construct_vocabulary.source_content_sha256 is null
+                  or occupational_construct_vocabulary.source_content_sha256 = excluded.source_content_sha256
+              )
+            returning vocabulary_id
+            """,
+            ONET_VOCABULARY_IRI,
+            ONET_RELEASE,
+            ONET_LICENSE_IRI,
+            ONET_ATTRIBUTION,
+            source_sha256,
+        )
+        if vocabulary_id is None:
+            raise ValueError("O*NET release metadata conflicts with the stored catalog")
+        await conn.executemany(
+            """
+            insert into occupational_construct
+                (vocabulary_id, construct_iri, construct_family_code,
+                 preferred_label, construct_description)
+            values ($1, $2, $3, $4, $5)
+            on conflict (vocabulary_id, construct_iri) do update set
+                construct_description = coalesce(
+                    occupational_construct.construct_description,
+                    excluded.construct_description
+                )
+            where occupational_construct.construct_family_code = excluded.construct_family_code
+              and occupational_construct.preferred_label = excluded.preferred_label
+              and (
+                  occupational_construct.construct_description is null
+                  or occupational_construct.construct_description = excluded.construct_description
+              )
+            """,
+            [
+                (
+                    vocabulary_id,
+                    construct.construct_iri,
+                    construct.family_code,
+                    construct.preferred_label,
+                    construct.description,
+                )
+                for construct in constructs
+            ],
+        )
+        rows = await conn.fetch(
+            """
+            select construct_iri, construct_family_code, preferred_label,
+                   construct_description
+              from occupational_construct
+             where vocabulary_id = $1
+            """,
+            vocabulary_id,
+        )
+        stored = {
+            str(row["construct_iri"]): (
+                str(row["construct_family_code"]),
+                str(row["preferred_label"]),
+                row["construct_description"],
+            )
+            for row in rows
+        }
+        expected = {
+            construct.construct_iri: (
+                construct.family_code,
+                construct.preferred_label,
+                construct.description,
+            )
+            for construct in constructs
+        }
+        if stored != expected:
+            raise ValueError("stored O*NET catalog differs from the official release")
+    return len(constructs)

migrations/0239_occupational_construct_catalog.sql

@@ -0,0 +1,25 @@
+-- ADR 0250: preserve the official release document and construct descriptions.
+
+alter table occupational_construct_vocabulary
+    add column if not exists source_content_sha256 text;
+
+alter table occupational_construct_vocabulary
+    drop constraint if exists occupational_construct_vocabulary_source_content_sha256_check;
+alter table occupational_construct_vocabulary
+    add constraint occupational_construct_vocabulary_source_content_sha256_check
+    check (
+        source_content_sha256 is null
+        or source_content_sha256 ~ '^[0-9a-f]{64}$'
+    );
+
+alter table occupational_construct
+    add column if not exists construct_description text;
+
+alter table occupational_construct
+    drop constraint if exists occupational_construct_description_nonblank_check;
+alter table occupational_construct
+    add constraint occupational_construct_description_nonblank_check
+    check (
+        construct_description is null
+        or btrim(construct_description) <> ''
+    );