Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
8a9e564
feat(ontology): publish O*NET 31 content model
seonghobae Aug 26, 2026
f566a08
Merge remote-tracking branch 'origin/feat/soc-2018-full-hierarchy' in…
seonghobae Aug 26, 2026
8ab2248
docs(gaps): refresh occupational PR gates
seonghobae Aug 26, 2026
82153e5
Merge remote-tracking branch 'origin/feat/soc-2018-full-hierarchy' in…
seonghobae Aug 26, 2026
38812fc
fix(ontology): sort O*NET outline numerically
seonghobae Aug 26, 2026
9a4a560
fix(ontology): order O*NET outline numerically
Aug 26, 2026
f42c42a
feat(ontology): publish O*NET content linkages
seonghobae Aug 26, 2026
2c84837
Merge commit 'refs/codex/pr731-overwritten' into HEAD
Aug 26, 2026
314f36a
Merge remote-tracking branch 'origin/feat/onet-31-content-model-ontol…
seonghobae Aug 26, 2026
c9b6e04
Merge remote-tracking branch 'origin/feat/onet-31-content-model-ontol…
seonghobae Aug 26, 2026
b2dd06f
chore(ontology): make linkage renderer executable
seonghobae Aug 26, 2026
1c62254
docs(gaps): record O*NET linkage stack
seonghobae Aug 26, 2026
88ae579
fix(adr): reserve O*NET content model decision number
Aug 26, 2026
9e714f6
Merge remote-tracking branch 'origin/feat/onet-31-content-model-ontol…
Aug 26, 2026
7cee035
docs(adr): resolve O*NET publication boundary
Aug 26, 2026
8d86b0e
Merge remote-tracking branch 'origin/feat/onet-31-content-model-ontol…
Aug 26, 2026
64018df
Merge remote-tracking branch 'origin/feat/soc-2018-full-hierarchy' in…
seonghobae Aug 26, 2026
9654bb8
Merge remote-tracking branch 'origin/feat/onet-31-content-model-ontol…
seonghobae Aug 26, 2026
3e30402
Merge remote-tracking branch 'origin/feat/onet-31-content-model-ontol…
seonghobae Aug 26, 2026
b3b9b36
Merge commit '1d2f80520b0735fe58d32350e2f839b64c626afe' into HEAD
seonghobae Aug 26, 2026
7f60aa8
Merge commit 'b3b9b36054d242ac7caa955a96b6ad98560dc9a9' into HEAD
seonghobae Aug 26, 2026
8dd5adb
feat(ontology): define O*NET occupation rating store (#734)
seonghobae Aug 26, 2026
0ae2f69
Merge remote-tracking branch 'origin/feat/soc-2018-full-hierarchy' in…
seonghobae Aug 26, 2026
8d50f45
fix(ontology): enforce O*NET linkage domains
Aug 26, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ flowchart LR
| `commitment_extraction.py` | Pluggable LLM derivation of a customer commitment (promise + deadline) from a post; `Null` default, `ContextualOrchestrator` real impl |
| `temporal_expressions.py` | Pure Korean relative-time resolver for Global Ask (ADR 0150) |
| `ask_time_axis.py` | Event-time vs ingestion-time clock choice for that window (ADR 0202) |
| `ontology.py` | Loads the governed Turtle source tree (`lineageweave-kg.ttl` plus generated fragments), the formal OWL 2/RDFS/SKOS vocabulary for the Knowledge Graph's node/edge types and source taxonomies (ADR 0004, ADR 0252, ADR 0255) |
| `ontology.py` | Loads the governed Turtle source tree (`lineageweave-kg.ttl` plus generated fragments), the formal OWL 2/RDFS/SKOS vocabulary for the Knowledge Graph's node/edge types, source taxonomies, and published O*NET linkages (ADR 0004, ADR 0252, ADR 0255, ADR 0256) |
| `ontology_neighborhood.py` | Bounded typed ontology/provenance neighborhood (ADR 0184); PostgreSQL stays authoritative, OWL subclass is not an instance edge |
| `ontology_source_cursor.py` | Opaque HMAC source-window continuation (ADR 0124); keyset pagination, never OFFSET |
| `period_report.py` | Fit GRM/GPCM on persisted IRT rows, FIPC-select, EAP-score a period (ADR 0003 slice 3; Bock & Mislevy, 1982) |
Expand Down
12 changes: 12 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,18 @@ All notable changes to this project are documented here. Format follows

### Added

- O*NET occupation-rating source evidence now has a replay-safe PostgreSQL
contract with normalized release, source-table, scale, occupation, element,
and observation tables. Exact release/source LIST partitions fail closed;
nullable categories remain idempotent identities, divergent duplicates and
truncation fail closed, and no source value is promoted to a local weight or
person score. Task Ratings remain outside this content-model-element store
pending their own normalized identity contract (ADR 0257).
- All eight O*NET 31.0 published linkage tables now contribute 1,417 directed
Ability/Essential Skill/Transferable Skill/Work Style relations to Work
Activities and Work Context. Every direct relation has an exact reified
PROV source-table assertion; no confidence, causal effect, rating, or weight
is inferred (ADR 0256).
- The complete O*NET 31.0 Content Model Reference is now a deterministic
3,006-concept SKOS fragment with exact source IDs, names, descriptions, and
documented outline parents (ADR 0255). Its read model fails closed on
Expand Down
44 changes: 44 additions & 0 deletions docs/adr/0256-onet-content-model-published-linkages.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# ADR 0256: O*NET content-model published linkages

**Status:** Accepted
**Date:** 2026-08-27
**Extends:** ADR 0248, ADR 0255

## Context

O*NET 31.0 publishes eight explicit linkage tables connecting Abilities,
Essential Skills, Transferable Skills, and Work Styles to relevant Work
Activities and Work Context elements. These 1,417 source pairs provide an
authoritative relationship layer without deriving relations from labels,
embeddings, local judgment, or arbitrary weights.

## Decision

1. Publish all eight tables and all 1,417 unique directed pairs. Preserve the
source and target O*NET element IDs and names exactly and fail generation
when either endpoint is absent from the pinned Content Model Reference.
2. Assert `:relevantWorkActivity` or `:relevantWorkContext` from the worker
element to the job element. Direction is preserved; the predicates are not
symmetric, causal, equivalent, or psychometric weights.
3. Reify every direct assertion as an `rdf:Statement` and `prov:Entity` with
exact `rdf:subject`, `rdf:predicate`, `rdf:object`, and
`prov:wasDerivedFrom` pointing to the one pinned O*NET table that published
it. No unsupported confidence or generated time is added.
4. Pin each official O*NET 31.0 JSON artifact and reproduce one deterministic
Turtle fragment. A duplicate pair inside a source table, endpoint-name
mismatch, unknown endpoint, wrong domain, or wrong row count fails closed.
5. Keep occupation-specific ratings separate. This decision imports no
occupation code, data value, scale, sample size, error, confidence interval,
suppression flag, person trait, or locally normalized score.

## Consequences

The semantic layer can traverse published cognition/style-to-behavior/context
relations with assertion-level provenance. Occupation ratings remain the next
separate import contract because their scales, uncertainty, suppression, and
update dates cannot be represented as these unweighted vocabulary links.

## Reference

National Center for O*NET Development. (2026). *O*NET 31.0 database: Content
model linkages* [Data set]. https://www.onetcenter.org/dictionary/31.0/json/
80 changes: 80 additions & 0 deletions docs/adr/0257-onet-occupation-rating-observation-store.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
# ADR 0257: O*NET occupation-rating observation store

**Status:** Accepted
**Date:** 2026-08-27
**Extends:** ADR 0166, ADR 0255, ADR 0256

## Context

O*NET 31.0 publishes occupation-specific ratings for content-model abilities, essential and
transferable skills, knowledge, education, training and experience, interests,
work styles, work activities, work context, and adjacent content-model
domains. A rating is not merely an edge: its meaning depends on the release,
source table, occupation, element, scale, optional response category, sample
size, standard error, 95% confidence interval, precision-suppression flag,
relevance flag, source update month, and domain source. O*NET publishes that
field as a seven-character `MM/YYYY` value; coercing it to a database date
would invent a day (National Center for O*NET Development, 2026b).

Flattening those attributes into ontology predicates would erase measurement
and provenance boundaries. Loading them into Python mathematical code would
also violate LineageWeave's externalized-compute boundary.

## Decision

1. Store source releases, source tables, scales, occupations, content-model
elements, and rating observations in separate third-normal-form tables.
2. Preserve published numeric values exactly as decimal observations. They are
source ratings, never locally estimated weights, person scores, causal
effects, or calibrated psychometric parameters.
3. An observation key is release + source table + occupation + element + scale
+ optional category. PostgreSQL `UNIQUE NULLS NOT DISTINCT` keeps an absent
category an honest single absence rather than replacing it with a sentinel.
4. Partition observations first by release and then by source-table code.
These are authoritative lifecycle/query boundaries and require no invented
hash modulus. An importer must create both exact LIST partitions before
inserting; without them PostgreSQL rejects the row.
5. Every insert uses the owning release/table artifact digest and idempotent
`ON CONFLICT DO NOTHING`. An exact duplicate is idempotent, while a row with
the same identity and different source values fails closed. Endpoint names
and scale names must match their normalized reference
rows; each scale definition names its owning source-table artifact, and the
importer rejects disagreement rather than overwriting identity.
6. Preserve `recommend_suppress` and `not_relevant` independently. A suppressed
value remains stored with its warning; a not-relevant value is not converted
to zero. Missing `n`, error, or interval values remain null.
7. Range and uncertainty constraints reject negative sample/error values,
inverted confidence intervals, malformed or future source update months, malformed source
digests, and values outside their declared scale bounds before persistence. Scale bounds
govern the published `Data Value`, not the optional response-category code. In particular,
O*NET's `CXP` rows store a category in `Category` while `Data Value` is the percentage that
endorsed it, so the authoritative `CXP` bounds are 0 through 100 (National Center for
O*NET Development, 2026b).
8. This content-model-element store excludes Task Ratings, whose integer Task
IDs and task-statement identity require a separate normalized target table;
it does not reinterpret a Task ID as a content-model element.
9. This store is immutable source evidence. Row mutation and whole-store
truncation fail closed. Any later aggregation, comparison,
temporal model, multilevel model, or occupational recommendation belongs to
TEPP/fast-mlsirm or another owning Rust service and must cite these rows.

## Consequences

LineageWeave can import the governed public O*NET content-model rating corpus without
manufacturing semantics or embedding large production datasets in git.
Release/source partitions localize hot imports and permit exact detach/archive
operations. A separate API/UI ADR is still required before exposing ratings.

## References

National Center for O*NET Development. (2026a). *O*NET 31.0 database* [Data
set]. https://www.onetcenter.org/database.html

National Center for O*NET Development. (2026b). *Work context: O*NET 31.0 data
dictionary*. https://www.onetcenter.org/dictionary/31.0/excel/work_context.html

PostgreSQL Global Development Group. (2026). *PostgreSQL 18 documentation:
Table partitioning*. https://www.postgresql.org/docs/current/ddl-partitioning.html

PostgreSQL Global Development Group. (2026). *PostgreSQL 18 documentation:
CREATE TABLE*. https://www.postgresql.org/docs/current/sql-createtable.html
2 changes: 2 additions & 0 deletions docs/adr/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,8 @@ decision from them.
| [`OCCUPATIONAL_CONSTRUCT_REFERENCES.md`](../doctoring/OCCUPATIONAL_CONSTRUCT_REFERENCES.md) | [0248](0248-occupational-construct-evidence-boundary.md) |
| [`ONET_31_CONTENT_MODEL_REFERENCES.md`](../doctoring/ONET_31_CONTENT_MODEL_REFERENCES.md) | [0255](0255-onet-31-content-model-ontology.md) |
| [`SOC_2018_HIERARCHY_REFERENCES.md`](../doctoring/SOC_2018_HIERARCHY_REFERENCES.md) | [0252](0252-complete-2018-soc-hierarchy.md) |
| [`ONET_31_LINKAGE_REFERENCES.md`](../doctoring/ONET_31_LINKAGE_REFERENCES.md) | [0256](0256-onet-content-model-published-linkages.md) |
| [`ONET_RATING_STORE_REFERENCES.md`](../doctoring/ONET_RATING_STORE_REFERENCES.md) | [0257](0257-onet-occupation-rating-observation-store.md) |

[0011](0011-prov-o-standard-relations.md) and [0065](0065-prov-o-provenance-boundary.md) cite the dated W3C PROV-O and PROV-DM Recommendations (https://www.w3.org/TR/2013/REC-prov-o-20130430/ and https://www.w3.org/TR/2013/REC-prov-dm-20130430/).

Expand Down
25 changes: 25 additions & 0 deletions docs/doctoring/ONET_31_LINKAGE_REFERENCES.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# O*NET 31.0 content-model linkage evidence

This supporting note records the eight official JSON artifacts governed by
ADR 0256. It introduces no independent architecture decision.

| Table | Rows | SHA-256 |
|---|---:|---|
| Abilities to Work Activities | 381 | `e03c3db1dbe8c7818198c943c8640cc2ddaf351972a79756b535940c84acfc52` |
| Abilities to Work Context | 139 | `7d5fbdc3df973790f1f24f98ace1f77c6cc8dab97197444e9ebf7695b6f59348` |
| Essential Skills to Work Activities | 110 | `35d389bb2a578d64bf49ef263f24cb5c1bb59c214bcec9045e476008023205f5` |
| Essential Skills to Work Context | 39 | `883f99fe723610530666031da9a68d72790f04cc87e0ac808f4600f1e01b8e5f` |
| Transferable Skills to Work Activities | 122 | `0b1755becb3100d4c19660cea2adf8d7c47a0654087560ceffd71c515fe94be2` |
| Transferable Skills to Work Context | 57 | `4add628b7257f2a1b7b204b1ddd715b82685559d35b020e540b15ca74abb2dd6` |
| Work Styles to Work Activities | 303 | `889071d37a3bb073854c72dc31f96a8e9079a18c4e581e6f5bf2e082f58c641b` |
| Work Styles to Work Context | 266 | `9e3aa088aa55d1f927688c55bb77987f5825ba0fdc9da3edebcb81cbf198fa54` |

The 1,417 rows contain identifiers and names only. They publish relevance
linkages, not coefficients, ratings, causal effects, or person scores. The
checked-in Turtle SHA-256 is
`6caff28f75cf6bd0a78548e85d4cd5217a77bd747c094e62f59b9cd389d36489`.

## APA 7 reference

National Center for O*NET Development. (2026). *O*NET 31.0 database: Content
model linkages* [Data set]. https://www.onetcenter.org/dictionary/31.0/json/
26 changes: 26 additions & 0 deletions docs/doctoring/ONET_RATING_STORE_REFERENCES.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# O*NET occupation-rating store evidence

This supporting note records the source and database capabilities governed by
ADR 0257. It introduces no independent architecture decision.

O*NET 31.0 occupation data tables publish occupation and element identifiers,
scale identifiers and names, decimal values, optional category values, sample
sizes, standard errors, confidence bounds, suppression/relevance flags, update
dates, and domain sources. These fields remain source observations; they are
not locally estimated psychometric weights.

PostgreSQL declarative partitioning provides exact LIST boundaries and
partition pruning. A unique constraint on a partitioned table includes its
partition key; `UNIQUE NULLS NOT DISTINCT` makes a missing category one stable
identity without a sentinel value.

## APA 7 references

National Center for O*NET Development. (2026). *O*NET 31.0 database* [Data
set]. https://www.onetcenter.org/database.html

PostgreSQL Global Development Group. (2026). *PostgreSQL 18 documentation:
Table partitioning*. https://www.postgresql.org/docs/current/ddl-partitioning.html

PostgreSQL Global Development Group. (2026). *PostgreSQL 18 documentation:
CREATE TABLE*. https://www.postgresql.org/docs/current/sql-createtable.html
Loading