Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ flowchart LR
| `commitment_extraction.py` | Pluggable LLM derivation of a customer commitment (promise + deadline) from a post; `Null` default, `ContextualOrchestrator` real impl |
| `temporal_expressions.py` | Pure Korean relative-time resolver for Global Ask (ADR 0150) |
| `ask_time_axis.py` | Event-time vs ingestion-time clock choice for that window (ADR 0202) |
| `ontology.py` | Loads `docs/ontology/lineageweave-kg.ttl`, the formal OWL 2/RDFS/SKOS vocabulary for the Knowledge Graph's node/edge types (ADR 0004) |
| `ontology.py` | Loads the governed Turtle source tree (`lineageweave-kg.ttl` plus generated fragments), the formal OWL 2/RDFS/SKOS vocabulary for the Knowledge Graph's node/edge types and source taxonomies (ADR 0004, ADR 0252, ADR 0255) |
| `ontology_neighborhood.py` | Bounded typed ontology/provenance neighborhood (ADR 0184); PostgreSQL stays authoritative, OWL subclass is not an instance edge |
| `ontology_source_cursor.py` | Opaque HMAC source-window continuation (ADR 0124); keyset pagination, never OFFSET |
| `period_report.py` | Fit GRM/GPCM on persisted IRT rows, FIPC-select, EAP-score a period (ADR 0003 slice 3; Bock & Mislevy, 1982) |
Expand Down Expand Up @@ -654,7 +654,8 @@ vocabulary (`node_type`, `edge_type`, `entity_relationship_type`,
`person_side`, `corporate_entity_level`) actually matches what the
Ontology/Semantic-Layer claim implies.

`docs/ontology/lineageweave-kg.ttl` is a real OWL 2 / RDFS / SKOS
`docs/ontology/lineageweave-kg.ttl` and its deterministic governed fragments
are a real OWL 2 / RDFS / SKOS
ontology in Turtle syntax: classes for `Post`/`Person`/`CorporateEntity`
(with `OurSidePerson`/`CounterpartyPerson` subclasses), object
properties for each `edge_type_code` and `entity_relationship_type`
Expand All @@ -668,7 +669,7 @@ specification over it, in the same sense W3C's own stack uses "semantic
layer" (RDFS/OWL as the governed conceptual layer over raw data), not a
separate BI-metrics product and not a parallel triple store.

`lineageweave/ontology.py` parses the Turtle file once with `rdflib`
`lineageweave/ontology.py` parses the Turtle source tree once with `rdflib`
(pure Python, no Rust toolchain, unlike `fast-mlsirm`) and exposes the
vocabulary as importable IRI constants, so application code has one
canonical name per class/property instead of re-typing lookup codes as
Expand Down
26 changes: 26 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,32 @@ All notable changes to this project are documented here. Format follows

### Added

- The complete O*NET 31.0 Content Model Reference is now a deterministic
3,006-concept SKOS fragment with exact source IDs, names, descriptions, and
documented outline parents (ADR 0255). Its read model fails closed on
malformed hierarchy data; no occupation rating, person trait, score,
weight, or inferred crosswalk is introduced.
- The complete 2018 SOC hierarchy is now published as a deterministic
ontology fragment: 23 major groups, 98 minor groups, 459 broad occupations,
and 867 detailed occupations retain their official codes, titles, levels,
and source-declared parents (ADR 0252). The read model supports fail-closed
lookup across all 1,447 nodes; no employer job-family mapping, weight, or
person-level inference is introduced.
- The occupational-classification and worker-characteristic taxonomy is now
published in the canonical ontology: all 23 major groups of the 2018
Standard Occupational Classification (the O*NET job families) carry
official titles and codes verbatim, the four O*NET 31.0 job-zone categories
carry their published names and source values, and source-native
worker-characteristic families are addressable -- Fleishman's four ability domains,
Holland's six RIASEC interest types with the published hexagonal adjacency,
the six legacy O*NET work-value clusters, and the seven higher-order
revised O*NET Work Styles dimensions (ADR 0245). Typed derivation properties from
classifications to characteristics are declared but assert no instance
binding; a deterministic application read model
(`lineageweave.io_taxonomy`) exposes fail-closed lookups, and no numeric
importance or level rating is imported. Each scheme links to versioned
PROV source entities with publisher/creator and rights/license metadata;
the stable O*NET 31.0 Job Zone JSON carries its verified SHA-256.
- Evidence-bound occupational construct semantics now keep cognitive
abilities, work styles, work activities, affective reactions, performance
behaviors, and FJA worker functions distinct. Record-to-construct links
Expand Down
15 changes: 10 additions & 5 deletions docs/adr/0159-published-ontology-pages.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,12 +40,17 @@ and consumer migration plan.
3. Publish equivalent machine-readable artifacts beside the HTML:
`ontology.ttl`, `ontology.jsonld`, `ontology.nt`, the PROV-O support profile,
and a source-digest manifest.
4. Preserve `lineageweave-kg.ttl` byte-for-byte as the published Turtle
artifact. JSON-LD and N-Triples are generated from a canonicalized RDF graph
and are tested for semantic isomorphism with the source.
4. For a single governed Turtle source, preserve `lineageweave-kg.ttl`
byte-for-byte as the published Turtle artifact. When a later accepted ADR
adds governed fragments, publish the merged graph as deterministic canonical
RDF that parses as Turtle; test every machine format for semantic
isomorphism with the complete source graph. Never concatenate independent
Turtle documents because their prefix and base declarations have
document-local scope.
5. Do not add a build timestamp. The same source tree must produce the same
artifact bytes. The manifest records the source SHA-256 and the complete
published ontology-directory inventory instead.
artifact bytes. The manifest records every governed source path and SHA-256,
the ordered source-tree SHA-256, and the complete published
ontology-directory inventory instead.
6. Run publication through `scripts/publish_ontology_site.py`, a fail-closed
boundary that rejects duplicate HTML fragments, non-HTTP(S) linked IRIs,
symlink outputs, source-overlapping outputs, and replacement of directories
Expand Down
160 changes: 160 additions & 0 deletions docs/adr/0245-io-occupational-taxonomy-in-the-published-ontology.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,160 @@
# ADR 0245: Occupational classification and worker-characteristic taxonomy in the published ontology

**Status:** Accepted
**Date:** 2026-08-26
**Extends:** [ADR 0004](0004-knowledge-graph-ontology.md), [ADR 0145](0145-psychometric-channel-weight-estimation.md), [ADR 0207](0207-repository-case-ontology-namespace-canonical.md), [ADR 0232](0232-worker-function-taxonomy-in-the-published-ontology.md)
**Superseded in part by:** [ADR 0252](0252-complete-2018-soc-hierarchy.md), which expands the major-group-only scheme into the complete 2018 SOC hierarchy.

## Context

Industrial and organizational psychology classifies work twice: once by
*what* the occupation is -- the 2018 Standard Occupational Classification
major groups, which the O*NET program publishes as its job families --
and once by *which human characteristics* doing it exercises. The O*NET
content model organizes those characteristics under every occupation as
abilities, occupational interests, work values, and work styles
(Peterson et al., 1999; Peterson et al., 2001). Holland's RIASEC hexagon
supplies the interest vocabulary with a published structural claim about
adjacency (Holland, 1997); Hogan and Holland (2003) established the
relationship between personality and job performance but is not the source
of the O*NET vocabulary; the revised O*NET Work Styles report supplies its
seven higher-order dimensions. Fleishman and Quaintance (1984) supply the
four ability domains. O*NET 31.0 publishes four Job Zone categories under
source values 2 through 5 after combining the former first two zones.

ADR 0232 already publishes the DOT/FJA worker functions, so stored
evidence can name *how* a worker functioned on data, people, or things.
It cannot yet say *which occupational family* the evidence belongs to,
nor resolve its cognitive, affective, and behavioral content into the
published characteristic families. The ontology is again the correct
home, under the same two constraints that bound ADR 0232:

1. The lookup-code round trip (`tests/test_ontology.py`) must stay
untouched: these concepts are not `common_lookup_value` rows.
2. Measurement stays governed by ADR 0145: no numeric importance or
level rating from any occupational profile may be imported, fitted,
or renormalized.

## Decision

1. Publish all 23 major groups of the 2018 Standard Occupational
Classification verbatim -- official titles and ``NN-0000`` codes --
as a `skos:ConceptScheme` (`:socMajorGroupScheme`) of
`:OccupationalMajorGroup` concepts, matching the O*NET job-family
grouping.
2. Publish the four O*NET 31.0 Job Zone categories (`:jobZoneScheme`) with
their published names and source values (`:jobZoneLevel` 2-5). These
values are source identifiers, not fitted or ordinal weights.
3. Publish distinct source-native worker-characteristic families, without
collapsing them into cognition, affect, or behavior, as subclasses of
`:WorkerCharacteristic` inside one scheme
(`:workerCharacteristicScheme`):
- Fleishman's four ability domains (`:AbilityDomain`);
- Holland's six RIASEC interest types (`:InterestType`), each with
the standard Interest Profiler family description verbatim;
- the six historical O*NET work-value clusters (`:WorkValueCluster`),
explicitly labeled legacy because O*NET 31.0 no longer publishes the
Work Values branch;
- the seven higher-order dimensions in the revised O*NET Work Styles
structure (`:WorkStyleFamily`). The 21 lower-order dimensions and the
separate four-component occupation-level analysis remain an explicit
import gap; neither may be inferred from these family nodes.
4. Assert only the published structural relation between interest
types: `:riasecAdjacentTo`, a symmetric property whose six asserted
pairs are exactly the hexagonal ring edges Realistic-Investigative-
Artistic-Social-Enterprising-Conventional-Realistic (Holland, 1997).
Adjacency is a similarity ordering, not a score.
5. Declare `:OccupationalClassification` as the future common hierarchy
class and four domain/range-typed derivation properties --
`:occupationalAbilityDemand`, `:occupationalInterestProfile`,
`:occupationalValueOrientation`, and `:occupationalWorkStyleNorm` --
but assert **no instance binding**. Binding a major group to a
characteristic requires importing a versioned released source
profile (for example an O*NET database release) with provenance in
its own future decision; inventing per-family profiles here would
fabricate evidence.
6. Like ADR 0232, none of these concepts carries `:lookupCode`; ranks
and levels are scale positions from published tables and are never
used as weights; every IRI is minted in the canonical
repository-case namespace (ADR 0207).
7. The application read model lives in `lineageweave/io_taxonomy.py`:
cached, deterministically sorted records for each scheme;
well-formed-key lookups that return ``None`` for genuinely
undeclared codes or levels (honest unknown); fail-closed
``ValueError`` for malformed keys, malformed TTL declarations,
neighbors outside the closed RIASEC vocabulary, or a type without
exactly two neighbors.
8. Each concept scheme names its source entities through
`prov:wasDerivedFrom`. Source entities retain title, publisher or creator,
explicit release/version, source URL, and applicable rights or license.
`:sourceArtifactSha256` is present only when an exact stable artifact was
downloaded and hashed; the O*NET 31.0 Job Zone JSON is pinned to SHA-256
`f66d665a2e507c825a71aedb2c13ba22765e8259bc6c7fe5b3cdfd8105475a66`.
A dynamic page without a reproducible artifact carries no invented digest.

## Consequences

- Job families, job zones, and the full published
source-native characteristic-family vocabulary become addressable and
citable inside the semantic layer before any persistence decision exists.
- A classification never supports inferring an individual's cognition,
affect, personality, behavior, competence, suitability, or job
performance. Those uses require their own intended-use validity and
fairness evidence and are outside this decision.
- Per-major-group characteristic profiles remain deliberately absent:
the derivation properties make their future shape typed and
addressable without asserting anything the sources do not state at
this granularity.
- Adding DB-backed profile bindings later means one migration plus
provenance-bearing import of a released source database -- the
extension path is additive by construction, mirroring ADR 0232.
- `tests/test_io_taxonomy.py` pins the published titles, counts,
adjacency structure, closed vocabularies, and fail-closed lookups, so
drift toward invented constructs fails CI.

## Verification

- `tests/test_io_taxonomy.py`: completeness (23 groups, 4 zones, 6
types, 6 legacy clusters, 7 style dimensions, 4 ability domains), verbatim
official titles, code-shape validation, published hexagon adjacency,
deterministic ordering, canonical namespace, lookup round-trip
isolation, and fail-closed lookups.
- `tests/test_ontology.py` continues to pass unchanged: the round trip
sees no new lookup codes.
- Source-provenance tests require every new scheme to resolve to the declared
PROV entity and verify O*NET version, publisher, CC BY 4.0 license, artifact
digest, and SOC version/publisher/rights metadata.

## References

Fleishman, E. A., & Quaintance, M. K. (1984). *Taxonomies of human
performance: The description of human tasks*. Academic Press.

Hogan, J., & Holland, B. (2003). Using theory to evaluate personality
and job-performance relations: A socioanalytic interpretation.
*Journal of Applied Psychology, 88*(1), 100-112.
https://doi.org/10.1037/0021-9010.88.1.100

Holland, J. L. (1997). *Making vocational choices: A theory of
vocational personalities and work environments* (3rd ed.). Psychological
Assessment Resources.

Peterson, N. G., Mumford, M. D., Borman, W. C., Jeanneret, P. R.,
Fleishman, E. A., Levin, K. Y., Campion, M. A., Mayfield, M. S.,
Morgeson, F. P., Pearlman, K., Gowing, M. K., Lancaster, A. R., Silver,
M. B., & Dye, D. M. (2001). Understanding work using the Occupational
Information Network (O*NET): Implications for practice and research.
*Personnel Psychology, 54*(2), 451-492.
https://doi.org/10.1111/j.1744-6570.2001.tb00098.x

National Center for O*NET Development. (2024). *Revisiting the work
styles domain of the O*NET content model* (updated May 2026).
https://www.onetcenter.org/reports/Work_Styles_New.html

National Center for O*NET Development. (2026). *Job zone reference:
O*NET 31.0 database*.
https://www.onetcenter.org/dictionary/31.0/json/job_zone_reference.html

U.S. Department of Labor. (2018). *2018 Standard Occupational
Classification System*. Bureau of Labor Statistics.
https://www.bls.gov/soc/
50 changes: 50 additions & 0 deletions docs/adr/0252-complete-2018-soc-hierarchy.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
# ADR 0252: Complete 2018 SOC hierarchy as a generated ontology fragment

**Status:** Accepted
**Date:** 2026-08-27

## Context

ADR 0245 publishes only the 23 SOC major groups. That is insufficient for
occupation-level evidence: the official 2018 SOC contains four aggregation
levels and 1,447 classifications. A label-derived parent or a locally invented
job-family crosswalk would violate the repository's evidence boundary.

## Decision

1. Import the complete official 2018 SOC structure: 23 major groups, 98 minor
groups, 459 broad occupations, and 867 detailed occupations.
2. Preserve the source row's level and parent exactly. Publish `skos:broader`
only from that parent column; never derive hierarchy from code digits or
titles.
3. Keep the normalized source snapshot at
`docs/ontology/data/soc-2018-structure.csv` and generate
`docs/ontology/soc-2018-structure.ttl` deterministically. The source XLSX
SHA-256 is
`ade08af40923266f3a854842e888ca3e93c15b26a147c20a2b12a61f4c4f4077`;
the normalized CSV SHA-256 is
`7de1c9d4da14d8eeb95197974d9dc1989752ebda235dd234b1693f336891f68e`.
4. Treat SOC as a statistical occupational classification, not an employer's
job family, job series, position, person trait, or psychometric score. Those
bindings require separately authorized source assertions.
5. Runtime and publication loaders merge the governed Turtle fragments into
one graph. The public artifact remains one canonical ontology namespace and
is serialized from that merged graph, rather than concatenating independent
Turtle documents. Its manifest identifies and hashes every governed input.

## Consequences

Occupation evidence can address every official 2018 SOC level without an
invented mapping. The generated fragment is larger, but review remains bounded
by the pinned source digests, deterministic renderer, exact counts, parent
closure, and graph tests.

## References

U.S. Bureau of Labor Statistics. (2018). *2018 Standard Occupational
Classification system*. U.S. Department of Labor.
https://www.bls.gov/soc/2018/

U.S. Bureau of Labor Statistics. (2018). *Standard Occupational
Classification and coding structure, 2018 SOC*. U.S. Department of Labor.
https://www.bls.gov/soc/2018/soc_2018_class_and_coding_structure.pdf
51 changes: 51 additions & 0 deletions docs/adr/0255-onet-31-content-model-ontology.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
# ADR 0255: O*NET 31.0 content-model semantic layer

**Status:** Accepted
**Date:** 2026-08-27
**Extends:** ADR 0245, ADR 0248, ADR 0252
**Supersedes:** ADR 0248 decision 2 only, solely where it prohibits publishing
the complete O*NET vocabulary in LineageWeave IRIs

## Context

The high-level concepts in ADR 0245 cannot address most work-related
cognition, affect, behavior, requirements, activities, or context. O*NET 31.0
publishes 3,006 content-model elements with stable element IDs, names,
descriptions, and a documented outline hierarchy encoded in those IDs.
ADR 0248 preferred external O*NET RDF identifiers and prohibited a complete
local vocabulary. The version-pinned Content Model Reference is instead the
reproducible authority adopted here, so this later decision replaces only that
publication choice. ADR 0248's evidence-bound assertions, construct
separation, source-only cross-scheme links, and prohibitions on person-trait,
score, weight, equivalence, and causal inference remain normative.

## Decision

1. Publish all 3,006 Content Model Reference rows as SKOS concepts in a
versioned `:onet31ContentModelScheme`. These repository-case IRIs identify
the pinned 31.0 snapshot; they do not assert identity or equivalence with an
external O*NET RDF term.
2. Preserve `element_id`, name, and description verbatim. Derive
`skos:broader` only by O*NET's documented rule: remove the final
period-delimited outline segment. A missing parent fails generation.
3. Type each node by the six published root domains and, where present, the 18
published second-level branches. These types are navigation boundaries, not
psychometric scales or assertions about a person.
4. Keep Work Styles distinct from affective reactions and abilities distinct
from observed performance. Do not infer occupation ratings, person traits,
causal relations, weights, or crosswalks from this vocabulary.
5. Pin the exact O*NET 31.0 JSON artifact SHA-256
`db59c30e4240931edce59310f2747f5476f058984b55f58f72c6f29faa30186f`
and verify deterministic Turtle reproduction.

## Consequences

The semantic layer can address the complete published O*NET conceptual
vocabulary. Occupation-specific observations and the separately published
ability/skill/style-to-activity/context linkages remain provenance-bearing
data imports governed by later decisions.

## Reference

National Center for O*NET Development. (2026). *O*NET 31.0 database: Content
Model Reference*. https://www.onetcenter.org/dictionary/31.0/json/content_model_reference.html
Loading