Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
dcecf7e
feat(ontology): import pinned O*NET occupation ratings
seonghobae Aug 26, 2026
76b3ef2
fix(ontology): accept sparse O*NET model profiles
seonghobae Aug 26, 2026
437008e
Merge updated O*NET rating contract into importer
Aug 26, 2026
56d8cbf
Merge remote-tracking branch 'origin/feat/onet-occupation-ratings-con…
seonghobae Aug 26, 2026
3ee8b90
docs(gaps): refresh O*NET delivery stack
seonghobae Aug 26, 2026
1b229c6
fix(test): preserve credentialed importer DSN
seonghobae Aug 26, 2026
14cdb7b
Merge remote-tracking branch 'origin/feat/onet-occupation-ratings-con…
seonghobae Aug 26, 2026
c0f31f1
fix(import): reject password-only source URL userinfo
seonghobae Aug 26, 2026
9bc9e93
feat(ontology): expose occupation rating evidence
seonghobae Aug 26, 2026
8f5b87b
Merge remote-tracking branch 'origin/feat/onet-rating-importer' into …
seonghobae Aug 26, 2026
547b30a
Merge commit '4c3677afcd954b0f37b2e28ba53afd869e738663' into HEAD
seonghobae Aug 26, 2026
026ba80
Merge remote-tracking branch 'origin/feat/onet-rating-importer' into …
seonghobae Aug 26, 2026
b035599
fix: reject malformed O*NET CSV rows
seonghobae Aug 26, 2026
8932d19
Merge remote-tracking branch 'origin/feat/onet-rating-importer' into …
seonghobae Aug 26, 2026
b74a841
feat(ontology): add occupation evidence view
seonghobae Aug 26, 2026
5281435
Merge remote-tracking branch 'origin/feat/onet-rating-read-api' into …
seonghobae Aug 26, 2026
317205b
Merge remote-tracking branch 'origin/feat/onet-rating-read-api' into …
seonghobae Aug 26, 2026
3d1d317
docs(gaps): refresh exact PR queue
seonghobae Aug 26, 2026
2130bab
Merge remote-tracking branch 'origin/feat/onet-rating-read-api' into …
seonghobae Aug 26, 2026
5697189
fix(ui): bind occupation evidence pages
seonghobae Aug 26, 2026
e6a1d62
docs: keep occupation rating ADR lint-clean
Aug 26, 2026
e204209
feat(ontology): expose occupation rating evidence (#738)
seonghobae Aug 26, 2026
5826fb7
Merge remote-tracking branch 'origin/feat/onet-rating-importer' into …
seonghobae Aug 26, 2026
4f7b928
fix(ui): validate occupation evidence links
Aug 26, 2026
3074246
Merge remote-tracking branch 'origin/feat/onet-rating-importer' into …
seonghobae Aug 26, 2026
b8bc5cf
Merge remote-tracking branch 'origin/feat/onet-rating-evidence-ui' in…
seonghobae Aug 26, 2026
2209ba6
Merge remote-tracking branch 'origin/feat/onet-rating-read-api' into …
Aug 26, 2026
b24e8fa
fix(ui): fence superseded occupation evidence
Aug 26, 2026
e6efa6f
fix(ui): hide stale occupation pagination
Aug 26, 2026
8dd5adb
feat(ontology): define O*NET occupation rating store (#734)
seonghobae Aug 26, 2026
3d3463e
Merge remote-tracking branch 'origin/feat/onet-content-model-linkages…
seonghobae Aug 26, 2026
97ad933
Merge branch 'feat/onet-rating-importer' into feat/onet-rating-eviden…
seonghobae Aug 26, 2026
36aa851
Merge commit '5fbdb9c326574edbd051be0daea1fa0f313edec5' into HEAD
Aug 26, 2026
e9cdf42
feat(ontology): catalog imported rating sources (#743)
seonghobae Aug 27, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,8 @@ flowchart LR
| `temporal_expressions.py` | Pure Korean relative-time resolver for Global Ask (ADR 0150) |
| `ask_time_axis.py` | Event-time vs ingestion-time clock choice for that window (ADR 0202) |
| `ontology.py` | Loads the governed Turtle source tree (`lineageweave-kg.ttl` plus generated fragments), the formal OWL 2/RDFS/SKOS vocabulary for the Knowledge Graph's node/edge types, source taxonomies, and published O*NET linkages (ADR 0004, ADR 0252, ADR 0255, ADR 0256) |
| `backend/app/occupation_rating_ingestion.py` | Projects authenticated, bounded occupation-rating source evidence with exact decimal and artifact provenance semantics (ADR 0258) |
| `backend/app/occupation_rating_ingestion.py` | Projects authenticated occupation-rating evidence plus persisted source and represented-occupation catalogs (ADR 0258, ADR 0260, ADR 0261) |
| `frontend/src/components/OccupationRatingProfile.tsx` | Selects imported source and stored occupation title before reading exact Dashboard evidence, preserving absence, uncertainty, and warning semantics (ADR 0259–0261) |
| `ontology_neighborhood.py` | Bounded typed ontology/provenance neighborhood (ADR 0184); PostgreSQL stays authoritative, OWL subclass is not an instance edge |
| `ontology_source_cursor.py` | Opaque HMAC source-window continuation (ADR 0124); keyset pagination, never OFFSET |
| `period_report.py` | Fit GRM/GPCM on persisted IRT rows, FIPC-select, EAP-score a period (ADR 0003 slice 3; Bock & Mislevy, 1982) |
Expand Down
11 changes: 11 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,17 @@ All notable changes to this project are documented here. Format follows

### Added

- Each imported rating source now exposes its exact represented O*NET-SOC
code/title catalog, and the Dashboard uses that catalog instead of requiring
users to know or type an occupation code (ADR 0261).
- Occupation evidence source selection now comes from an authenticated catalog
of actually imported rating artifacts, with release, publisher, license,
digest, URL, and row-count provenance and fail-closed loading/empty/error
states (ADR 0260).
- The existing Dashboard now includes an authenticated occupation-evidence
view with exact source selection, sample/error context, textual suppression
warnings, artifact links, responsive table access, and distinct unavailable
versus empty next actions (ADR 0259).
- Authenticated occupation profiles can now read one exact imported
release/source with rating and scale artifact provenance, exact decimal
strings, uncertainty, suppression/relevance flags, and explicit unavailable
Expand Down
36 changes: 35 additions & 1 deletion backend/app/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,11 @@
upsert_commitment_ticket,
)
from backend.app.operations_dashboard import fetch_operations_dashboard
from backend.app.occupation_rating_ingestion import fetch_occupation_ratings
from backend.app.occupation_rating_ingestion import (
fetch_occupation_rating_sources,
fetch_occupation_ratings,
fetch_rating_source_occupations,
)
from backend.app.keyman_ingestion import ingest_post_keymen
from backend.app.knowledge_graph import (
corporate_entity_exists,
Expand Down Expand Up @@ -2302,6 +2306,36 @@ async def read_occupation_ratings(
)


@app.get("/api/occupation-rating-sources")
async def read_occupation_rating_sources(
_account: CurrentAccount = Depends(get_current_account),
pool: asyncpg.Pool = Depends(get_pool),
) -> dict[str, list[dict[str, object]]]:
"""Return the authenticated catalog of imported occupation-rating sources."""
async with pool.acquire() as conn:
return await fetch_occupation_rating_sources(conn)


@app.get("/api/occupation-rating-occupations")
async def read_rating_source_occupations(
data_release_code: str = Query(
..., min_length=1, max_length=63, pattern=r"^[a-z0-9][a-z0-9.-]*$"
),
source_table_code: str = Query(
..., min_length=1, max_length=63, pattern=r"^[a-z][a-z0-9_]*$"
),
_account: CurrentAccount = Depends(get_current_account),
pool: asyncpg.Pool = Depends(get_pool),
) -> dict[str, object]:
"""Return occupations represented in one imported rating source."""
async with pool.acquire() as conn:
return await fetch_rating_source_occupations(
conn,
data_release_code=data_release_code,
source_table_code=source_table_code,
)


@app.get("/api/posts/{post_id}/counterparties")
async def read_post_counterparties(
post_id: str,
Expand Down
71 changes: 71 additions & 0 deletions backend/app/occupation_rating_ingestion.py
Original file line number Diff line number Diff line change
Expand Up @@ -125,3 +125,74 @@ async def fetch_occupation_ratings(
"items": items,
"next_offset": offset + limit if len(rows) > limit else None,
}


async def fetch_occupation_rating_sources(
conn: RatingReadConnection,
) -> dict[str, list[dict[str, object]]]:
"""Return imported rating artifacts that contain at least one observation."""
rows = await conn.fetch(
"""select source.data_release_code, release.release_version,
release.source_publisher_name, release.source_license_url,
source.source_table_code, source.source_table_name,
source.source_artifact_url, source.source_artifact_sha256,
source.source_row_count
from occupational_source_table source
join occupational_data_release release
on release.data_release_code = source.data_release_code
where source.source_table_code <> 'scales_reference'
and exists (
select 1
from occupational_rating_observation observation
where observation.data_release_code = source.data_release_code
and observation.source_table_code = source.source_table_code
)
order by release.imported_at desc, source.data_release_code,
source.source_table_name, source.source_table_code"""
)
return {"sources": [dict(row) for row in rows]}


async def fetch_rating_source_occupations(
conn: RatingReadConnection,
*,
data_release_code: str,
source_table_code: str,
) -> dict[str, object]:
"""Return occupations with observations in one exact imported source."""
source = await conn.fetchrow(
"""select 1
from occupational_source_table
where data_release_code = $1 and source_table_code = $2""",
data_release_code,
source_table_code,
)
if source is None:
return {
"data_release_code": data_release_code,
"source_table_code": source_table_code,
"source_available": False,
"occupations": [],
}
rows = await conn.fetch(
"""select classification.onetsoc_code, classification.occupation_title
from occupational_classification_entry classification
where classification.data_release_code = $1
and exists (
select 1
from occupational_rating_observation observation
where observation.data_release_code = classification.data_release_code
and observation.source_table_code = $2
and observation.onetsoc_code = classification.onetsoc_code
)
order by classification.occupation_title,
classification.onetsoc_code""",
data_release_code,
source_table_code,
)
return {
"data_release_code": data_release_code,
"source_table_code": source_table_code,
"source_available": True,
"occupations": [dict(row) for row in rows],
}
52 changes: 52 additions & 0 deletions docs/adr/0259-occupation-rating-evidence-ui.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
# ADR 0259: Occupation-rating evidence in the existing Dashboard

- Status: Accepted
- Date: 2026-08-27
- Extends: ADR 0183, ADR 0206, ADR 0258
- Figma file ID: `1Su3lDRmiZdcUs47t1QwIX`

## Context

ADR 0258 makes an exact imported occupation profile readable, but an API does
not let an authenticated user find a published work characteristic or notice
that a value has low precision. ADR 0183 fixes the analyst GNB and prohibits a
new destination for every evidence type. The existing Dashboard is the place
for evidence-oriented next actions and already owns responsive table and form
tokens under ADR 0206.

## Decision

1. Add the occupation profile below the existing operations evidence on the
Dashboard. Do not add or rename a GNB destination.
2. Require the user to submit an exact O*NET-SOC code, data release, and source
table. Native form validation rejects malformed occupation codes before a
request; the API remains the trust-boundary validator.
3. Show each exact published value beside its declared scale bounds, optional
category, sample size, standard error, confidence interval, source month,
domain source, suppression warning, and not-relevant flag. Do not calculate
a score, rank, weight, trait estimate, or recommendation.
4. Keep `source unavailable` distinct from `occupation has no observations`.
Both states give a next action instead of displaying zero or a blank table.
5. Link the rating artifact and scale definition. The API carries their
digests and row counts for provenance; a later disclosure control may show
those identifiers when user research demonstrates that it aids the task.
6. Reuse the existing Dashboard Figma file, design tokens, native controls,
responsive overflow, focus behavior, and reduced-motion baseline. The table
has a named keyboard-focusable region and every warning is text, not color.
7. Storybook records populated, narrow, source-unavailable, and empty-profile
scenes using synthetic records only. Runtime screenshot review covers the
populated desktop and narrow scenes.

## Consequences

Users can inspect source evidence without confusing absence, low precision, or
not-relevant responses with a negative occupational conclusion. The interface
does not introduce a local psychometric or inference implementation.

## References

National Center for O*NET Development. (2026). *O*NET 31.0 database* [Data
set]. https://www.onetcenter.org/database.html

World Wide Web Consortium. (2024). *Web Content Accessibility Guidelines
(WCAG) 2.2* (W3C Recommendation). https://www.w3.org/TR/WCAG22/
41 changes: 41 additions & 0 deletions docs/adr/0260-occupation-rating-source-catalog.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# ADR 0260: Imported occupation-rating source catalog

- Status: Accepted
- Date: 2026-08-27
- Extends: ADR 0257, ADR 0258, ADR 0259

## Context

ADR 0259 initially requires users to type internal release and source-table
codes. That makes a valid product action depend on repository knowledge and
allows a user to request a source that was never imported. The normalized
rating store already owns the exact imported artifact catalog and therefore is
the only authoritative selector source.

## Decision

1. Add an authenticated read endpoint that lists rating artifacts containing
at least one persisted occupation observation. Exclude the Scales Reference
support artifact from selectable rating sources.
2. Return release code/version, publisher and license, source code/name, URL,
SHA-256, and declared row count. Order releases by persisted import time and
sources by stored name/code; do not infer recency from a version string.
3. The Dashboard selects only an entry returned by this endpoint. If the
catalog is loading, empty, or unavailable, disable profile submission and
give the user a next action. Do not retain a hidden hand-written fallback.
4. Authentication matches ADR 0258: imported O*NET artifacts are public
reference data, while the catalog still requires a valid workspace account.
5. The catalog does not claim that all official O*NET artifacts are imported.
It describes only current database state with immutable artifact provenance.

## Consequences

The occupation evidence workflow no longer asks users to know storage codes,
and an unavailable artifact cannot masquerade as a selectable source. Adding
an official artifact remains an importer operation with digest and row-count
validation rather than a UI-created catalog row.

## References

National Center for O*NET Development. (2026). *O*NET 31.0 database* [Data
set]. https://www.onetcenter.org/database.html
45 changes: 45 additions & 0 deletions docs/adr/0261-rating-source-occupation-selector.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# ADR 0261: Occupations represented by an imported rating source

- Status: Accepted
- Date: 2026-08-27
- Extends: ADR 0257, ADR 0258, ADR 0260

## Context

The source catalog removes internal release/source entry, but ADR 0260 still
leaves users to type an O*NET-SOC code. The normalized store already preserves
the source occupation title and code. A release may contain occupations that
are absent from one rating artifact, so the release classification alone is
not sufficient evidence that a profile exists for the selected source.

## Decision

1. Add an authenticated read endpoint returning stored O*NET-SOC code/title
pairs that have at least one observation in one exact imported rating
source. Keep unavailable source distinct from an available empty source.
2. Join by normalized release/code identity and an observation-existence
predicate. Do not bind occupations by title similarity, keyword inference,
external search, or a locally reconstructed classification.
3. Order by the stored occupation title and then code. Return the complete
represented set because the official imported classification is the
authoritative finite selector domain; do not introduce an arbitrary result
cutoff that makes valid occupations disappear.
4. Replace free-text occupation-code entry with a native select whose visible
label begins with the stored title and retains the exact code. Changing the
rating source clears both occupation selection and displayed evidence;
changing the occupation clears displayed evidence.
5. While the occupation catalog is loading, empty, or unavailable, disable
profile submission and state the next action. Pagination remains bound to
the identifiers returned by the loaded profile under ADR 0259.

## Consequences

Users choose an occupation by its authoritative title without knowing an
internal code, while API requests continue to carry exact stable identifiers.
Employer job families and series remain outside this selector until their
separate authorized import contract exists.

## References

National Center for O*NET Development. (2026). *O*NET 31.0 database* [Data
set]. https://www.onetcenter.org/database.html
3 changes: 3 additions & 0 deletions docs/adr/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,9 @@ decision from them.
| [`ONET_31_LINKAGE_REFERENCES.md`](../doctoring/ONET_31_LINKAGE_REFERENCES.md) | [0256](0256-onet-content-model-published-linkages.md) |
| [`ONET_RATING_STORE_REFERENCES.md`](../doctoring/ONET_RATING_STORE_REFERENCES.md) | [0257](0257-onet-occupation-rating-observation-store.md) |
| Occupation-rating authenticated read projection | [0258](0258-occupation-rating-read-api.md) |
| Occupation-rating Dashboard evidence view | [0259](0259-occupation-rating-evidence-ui.md) |
| Imported occupation-rating source catalog | [0260](0260-occupation-rating-source-catalog.md) |
| Rating-source occupation selector | [0261](0261-rating-source-occupation-selector.md) |

[0011](0011-prov-o-standard-relations.md) and [0065](0065-prov-o-provenance-boundary.md) cite the dated W3C PROV-O and PROV-DM Recommendations (https://www.w3.org/TR/2013/REC-prov-o-20130430/ and https://www.w3.org/TR/2013/REC-prov-dm-20130430/).

Expand Down
42 changes: 42 additions & 0 deletions docs/product-requirements.md
Original file line number Diff line number Diff line change
Expand Up @@ -161,6 +161,48 @@ Acceptance: invalid identifiers and unbounded pages are rejected; an unavailable
source never appears as a negative profile; pagination is deterministic; and a
suppressed observation retains its value and warning flag together.

### PRD-FR-2F — Occupation-rating evidence view

- Let an authenticated user submit an exact O*NET-SOC code, release, and source
from the existing Dashboard without changing the governed GNB (ADR 0259).
- Display published values beside bounds, sample/error/interval evidence,
source time, and text warnings; link both source artifacts.
- Give different next actions for unavailable source, empty occupation,
transport failure, and additional pages.

Acceptance: keyboard users can operate the form and named horizontally
scrollable table; narrow layouts retain complete values; suppression remains
visible beside its value; and Storybook covers populated, narrow, unavailable,
and empty states using synthetic data.

### PRD-FR-2G — Imported rating-source catalog

- Populate the occupation evidence selector only from imported artifacts that
contain observations, preserving release and artifact provenance (ADR 0260).
- Exclude the scale-definition support artifact from the rating-source selector.
- Disable profile submission and state the next action while the catalog is
loading, empty, or unavailable.

Acceptance: a user never types an internal release/source code; the selector
order follows persisted import time rather than parsed version heuristics; and
the real PostgreSQL integration test proves an imported synthetic artifact is
listed while its supporting scale artifact is not.

### PRD-FR-2H — Occupations represented in a rating source

- Populate the occupation selector with exact stored code/title pairs that
have observations in the selected imported source (ADR 0261).
- Clear the current occupation and profile when the source changes, and clear
the profile when the occupation changes; never mix continuation rows across
occupations or sources.
- Keep unavailable source, available-empty source, loading, and transport
failure distinct and actionable.

Acceptance: a user selects a stored title rather than typing an internal code;
the PostgreSQL integration test proves the source membership predicate; and
component tests prove selector changes clear prior evidence and pagination
stays bound to the loaded profile identifiers.

### PRD-FR-3 — Bounded ontology exploration

- Apply RBAC/ABAC, source eligibility, and knowledge cutoff before graph
Expand Down
Loading