Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,8 +80,8 @@ flowchart LR
| `temporal_expressions.py` | Pure Korean relative-time resolver for Global Ask (ADR 0150) |
| `ask_time_axis.py` | Event-time vs ingestion-time clock choice for that window (ADR 0202) |
| `ontology.py` | Loads the governed Turtle source tree (`lineageweave-kg.ttl` plus generated fragments), the formal OWL 2/RDFS/SKOS vocabulary for the Knowledge Graph's node/edge types, source taxonomies, and published O*NET linkages (ADR 0004, ADR 0252, ADR 0255, ADR 0256) |
| `backend/app/occupation_rating_ingestion.py` | Projects authenticated, bounded occupation-rating evidence and the persisted selectable-source catalog (ADR 0258, ADR 0260) |
| `frontend/src/components/OccupationRatingProfile.tsx` | Selects an imported source and reads exact occupation evidence in the existing Dashboard while preserving absence, uncertainty, and warning semantics (ADR 0259, ADR 0260) |
| `backend/app/occupation_rating_ingestion.py` | Projects authenticated, bounded occupation-rating evidence, the persisted selectable-source catalog, and occupations that have observations in a selected source (ADR 0258, ADR 0260, ADR 0261) |
| `frontend/src/components/OccupationRatingProfile.tsx` | Selects an imported source and published occupation title, then reads exact occupation evidence in the existing Dashboard while preserving absence, uncertainty, and warning semantics (ADR 0259, ADR 0260, ADR 0261) |
| `ontology_neighborhood.py` | Bounded typed ontology/provenance neighborhood (ADR 0184); PostgreSQL stays authoritative, OWL subclass is not an instance edge |
| `ontology_source_cursor.py` | Opaque HMAC source-window continuation (ADR 0124); keyset pagination, never OFFSET |
| `period_report.py` | Fit GRM/GPCM on persisted IRT rows, FIPC-select, EAP-score a period (ADR 0003 slice 3; Bock & Mislevy, 1982) |
Expand Down
6 changes: 6 additions & 0 deletions CHANGELOG.d/2.21.0-occupation-catalog.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
### Added

- Occupation evidence now selects a published occupation title from occupations
that actually have observations in the chosen imported source, with fail-closed
empty/unavailable catalog states and the retained title on the opened profile
(ADR 0261).
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,10 @@ All notable changes to this project are documented here. Format follows

### Added

- Occupation evidence now selects a published occupation title from occupations
that actually have observations in the chosen imported source, with fail-closed
empty/unavailable catalog states and the retained title on the opened profile
(ADR 0261).
- Occupation evidence source selection now comes from an authenticated catalog
of actually imported rating artifacts, with release, publisher, license,
digest, URL, and row-count provenance and fail-closed loading/empty/error
Expand Down
23 changes: 23 additions & 0 deletions backend/app/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,7 @@
)
from backend.app.operations_dashboard import fetch_operations_dashboard
from backend.app.occupation_rating_ingestion import (
fetch_occupation_rating_occupations,
fetch_occupation_rating_sources,
fetch_occupation_ratings,
)
Expand Down Expand Up @@ -2315,6 +2316,28 @@ async def read_occupation_rating_sources(
return await fetch_occupation_rating_sources(conn)


@app.get(
"/api/occupation-rating-sources/{data_release_code}/{source_table_code}/occupations"
)
async def read_occupation_rating_occupations(
data_release_code: str = Path(
..., min_length=1, max_length=63, pattern=r"^[a-z0-9][a-z0-9.-]*$"
),
source_table_code: str = Path(
..., min_length=1, max_length=63, pattern=r"^[a-z][a-z0-9_]*$"
),
_account: CurrentAccount = Depends(get_current_account),
pool: asyncpg.Pool = Depends(get_pool),
) -> dict[str, object]:
"""Return occupations that have observations in one imported rating source."""
async with pool.acquire() as conn:
return await fetch_occupation_rating_occupations(
conn,
data_release_code=data_release_code,
source_table_code=source_table_code,
)


@app.get("/api/posts/{post_id}/counterparties")
async def read_post_counterparties(
post_id: str,
Expand Down
73 changes: 72 additions & 1 deletion backend/app/occupation_rating_ingestion.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@
from decimal import Decimal
from typing import Any, Protocol

OCCUPATION_CATALOG_BOUND = 2000


class RatingReadConnection(Protocol):
"""Small asyncpg-compatible surface used by the rating read projection."""
Expand All @@ -21,6 +23,14 @@ def _decimal_text(value: Decimal | None) -> str | None:
return str(value) if value is not None else None


def _optional_mapping_value(row: Any, key: str) -> Any:
"""Return a mapping value when the projection supplied the column."""
try:
return row[key]
except (KeyError, IndexError):
return None


async def fetch_occupation_ratings(
conn: RatingReadConnection,
*,
Expand All @@ -38,21 +48,27 @@ async def fetch_occupation_ratings(
rating_source.source_row_count,
scale_source.source_artifact_url as scale_artifact_url,
scale_source.source_artifact_sha256 as scale_artifact_sha256,
scale_source.source_row_count as scale_source_row_count
scale_source.source_row_count as scale_source_row_count,
occupation.occupation_title
from occupational_source_table rating_source
left join occupational_source_table scale_source
on scale_source.data_release_code = rating_source.data_release_code
and scale_source.source_table_code = 'scales_reference'
left join occupational_classification_entry occupation
on occupation.data_release_code = rating_source.data_release_code
and occupation.onetsoc_code = $3
where rating_source.data_release_code = $1
and rating_source.source_table_code = $2""",
data_release_code,
source_table_code,
onetsoc_code,
)
if source is None:
return {
"data_release_code": data_release_code,
"source_table_code": source_table_code,
"onetsoc_code": onetsoc_code,
"occupation_title": None,
"source_available": False,
"source": None,
"items": [],
Expand Down Expand Up @@ -112,6 +128,7 @@ async def fetch_occupation_ratings(
"data_release_code": data_release_code,
"source_table_code": source_table_code,
"onetsoc_code": onetsoc_code,
"occupation_title": _optional_mapping_value(source, "occupation_title"),
"source_available": True,
"source": {
"source_table_name": source["source_table_name"],
Expand Down Expand Up @@ -151,3 +168,57 @@ async def fetch_occupation_rating_sources(
source.source_table_name, source.source_table_code"""
)
return {"sources": [dict(row) for row in rows]}


async def fetch_occupation_rating_occupations(
conn: RatingReadConnection,
*,
data_release_code: str,
source_table_code: str,
) -> dict[str, object]:
"""Return occupations that have observations in one imported rating source."""
source = await conn.fetchrow(
"""select source_table_code
from occupational_source_table
where data_release_code = $1
and source_table_code = $2
and source_table_code <> 'scales_reference'""",
data_release_code,
source_table_code,
)
if source is None:
return {
"data_release_code": data_release_code,
"source_table_code": source_table_code,
"source_available": False,
"occupations": [],
}
rows = await conn.fetch(
"""select occupation.onetsoc_code, occupation.occupation_title
from occupational_classification_entry occupation
where occupation.data_release_code = $1
and exists (
select 1
from occupational_rating_observation observation
where observation.data_release_code = occupation.data_release_code
and observation.source_table_code = $2
and observation.onetsoc_code = occupation.onetsoc_code
)
order by occupation.occupation_title, occupation.onetsoc_code
limit $3""",
data_release_code,
source_table_code,
OCCUPATION_CATALOG_BOUND,
)
return {
"data_release_code": data_release_code,
"source_table_code": source_table_code,
"source_available": True,
"occupations": [
{
"onetsoc_code": row["onetsoc_code"],
"occupation_title": row["occupation_title"],
}
for row in rows
],
}
46 changes: 46 additions & 0 deletions docs/adr/0261-occupation-rating-occupation-catalog.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# ADR 0261: Imported occupation catalog for rating evidence

- Status: Accepted
- Date: 2026-08-27
- Extends: ADR 0257, ADR 0258, ADR 0259, ADR 0260

## Context

ADR 0260 removed typed release and source-table codes, but occupation
selection still requires an exact O*NET-SOC code. That makes a valid product
action depend on repository knowledge and allows a user to request an
occupation that has no observation in the selected source. The normalized
store already owns `occupational_classification_entry` titles and the
observation identities that prove a source actually describes an occupation.

## Decision

1. Add an authenticated read endpoint that lists occupations with at least
one persisted observation in one exact imported rating source. Return the
official O*NET-SOC code and occupation title. Exclude occupations that
exist only as classification rows, and treat the Scales Reference support
artifact as unavailable.
2. Order by published title then code. Bound the catalog; do not rank,
recommend, or infer similarity. The catalog describes current database
state, not the complete official O*NET occupation list.
3. Distinguish an unavailable source from an available source with no
selectable occupation. Authentication matches ADR 0258.
4. The Dashboard occupation control is a native select populated only from
this catalog for the currently selected source. Changing the source
reloads and resets the occupation. If the occupation catalog is loading,
empty, or unavailable, disable profile submission and give the next
action. Do not retain a typed SOC fallback.
5. Display the published occupation title with the retained code on the
opened profile. Derive no ranking or recommendation.

## Consequences

Users select a published occupation that the chosen source actually
describes. An unavailable or empty occupation catalog cannot masquerade as a
typed code. Adding an official occupation remains an importer operation with
digest and row-count validation rather than a UI-created catalog row.

## References

National Center for O*NET Development. (2026). *O*NET 31.0 database* [Data
set]. https://www.onetcenter.org/database.html
1 change: 1 addition & 0 deletions docs/adr/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ decision from them.
| Occupation-rating authenticated read projection | [0258](0258-occupation-rating-read-api.md) |
| Occupation-rating Dashboard evidence view | [0259](0259-occupation-rating-evidence-ui.md) |
| Imported occupation-rating source catalog | [0260](0260-occupation-rating-source-catalog.md) |
| Imported occupation catalog for rating evidence | [0261](0261-occupation-rating-occupation-catalog.md) |

[0011](0011-prov-o-standard-relations.md) and [0065](0065-prov-o-provenance-boundary.md) cite the dated W3C PROV-O and PROV-DM Recommendations (https://www.w3.org/TR/2013/REC-prov-o-20130430/ and https://www.w3.org/TR/2013/REC-prov-dm-20130430/).

Expand Down
22 changes: 20 additions & 2 deletions docs/product-requirements.md
Original file line number Diff line number Diff line change
Expand Up @@ -163,8 +163,9 @@ suppressed observation retains its value and warning flag together.

### PRD-FR-2F — Occupation-rating evidence view

- Let an authenticated user submit an exact O*NET-SOC code, release, and source
from the existing Dashboard without changing the governed GNB (ADR 0259).
- Let an authenticated user submit an imported source and a published
occupation from the existing Dashboard without changing the governed GNB
(ADR 0259, ADR 0260, ADR 0261).
- Display published values beside bounds, sample/error/interval evidence,
source time, and text warnings; link both source artifacts.
- Give different next actions for unavailable source, empty occupation,
Expand All @@ -188,6 +189,23 @@ order follows persisted import time rather than parsed version heuristics; and
the real PostgreSQL integration test proves an imported synthetic artifact is
listed while its supporting scale artifact is not.

### PRD-FR-2H — Imported occupation catalog

- Populate the occupation selector only from occupations that have
observations in the currently selected imported source, showing the
published title with the retained O*NET-SOC code (ADR 0261).
- Reset the occupation when the source changes.
- Disable profile submission and state the next action while the occupation
catalog is loading, empty, or unavailable; keep unavailable sources
distinct from an empty occupation list.
- Display the published title on the opened profile without deriving a
ranking or recommendation.

Acceptance: a user never types an O*NET-SOC code; occupation order follows
published title then code; and the real PostgreSQL integration test proves an
imported synthetic occupation is listed for its rating source while a
classification-only occupation is not.

### PRD-FR-3 — Bounded ontology exploration

- Apply RBAC/ABAC, source eligibility, and knowledge cutoff before graph
Expand Down
2 changes: 1 addition & 1 deletion docs/product-technical-gap-baseline.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ explicit unavailable state, not a reason to infer mappings from labels.
| Occupation-to-construct relations | ADR 0257 defines a candidate 3NF, release/source-partitioned immutable observation store and deterministic pinned-CSV importer preserving value, optional category, sample/error/CI, suppression, relevance, exact `MM/YYYY` source update month, source digest, and domain provenance. The official O*NET 31.0 Abilities file (94,640 rows, 910 occupations, 52 elements; SHA-256 `7e9cd79791ce6014e1d26d0a449ae5b1e7aa7ef52d39b3934c3bb8d438104b88`) and all 33 Scales Reference rows (SHA-256 `bcba23858ce21ecaacbde303a8993e35d46724b4afb8c9ec2b10e04f42adcfc9`) imported into a throwaway local PostgreSQL database with all 94,640 observations, 55 suppression flags, 7,572 not-relevant flags, and source months from `12/2004` through `08/2026`; every scale retained `scales_reference` artifact provenance, the database was dropped afterward, and no corpus is committed or claimed deployed | Pass exact-head review/checks and protected merge; validate and import every selected official rating artifact through an authorized runtime, returning only aggregate evidence; never invent or locally normalize a weight |
| Job-family and job-series semantics | No authoritative employer-specific job architecture is present | Define an organization-neutral import contract that preserves the authorized source hierarchy and distinguishes standard occupation codes from employer job families/series; no label-based binding |
| Temporal and multilevel interpretation | Static vocabulary only; no person-level inference is asserted | Version valid and transaction time, preserve occupation/organization/unit nesting and multiple membership, and require TEPP or the owning Rust psychometric service before any calibrated temporal or multilevel result |
| Product consumption | ADR 0258 defines a candidate authenticated occupation-rating API; ADR 0259 adds a candidate Dashboard evidence view with exact value/error/warning semantics; ADR 0260 replaces internal code entry with a candidate authenticated catalog of artifacts that actually contain observations. Component/API/PostgreSQL tests and Storybook build cover the current contract; synthetic populated scenes were visually audited at 1440×900 and 390×844. Protected delivery and authenticated runtime evidence remain absent | Pass exact-head review/checks and protected merge; verify the authenticated catalog, profile API, and rendered Dashboard against an authorized imported source using only aggregate/non-identifying evidence |
| Product consumption | ADR 0258 defines a candidate authenticated occupation-rating API; ADR 0259 adds a candidate Dashboard evidence view with exact value/error/warning semantics; ADR 0260 replaces internal source-code entry with a candidate authenticated catalog of artifacts that actually contain observations; ADR 0261 replaces typed O*NET-SOC entry with a candidate catalog of occupations that have observations in the selected source, showing published titles. Component/API/PostgreSQL tests and Storybook build cover the current contract; synthetic populated scenes were visually audited at 1440×900 and 390×844. Protected delivery and authenticated runtime evidence remain absent | Pass exact-head review/checks and protected merge; verify the authenticated source/occupation catalogs, profile API, and rendered Dashboard against an authorized imported source using only aggregate/non-identifying evidence |

### Current exact-head PR queue

Expand Down
2 changes: 1 addition & 1 deletion docs/storybook-inventory.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ operator-facing control you can click before changing product CSS.
| Story | Operator next action | Token / module |
|---|---|---|
| `Workspace/OperationsDashboard` | Compare Event and post counts, inspect external-information coverage, then open the cited source behind a claim, handover, or repeat-issue fact. `EvidenceReady`, `NarrowViewport`, `AnalysisPendingAndMissingEvidence`, `AnalysisFailed`, and `LoadError` cover populated, mobile, unavailable-evidence, analysis-pending, retryable failure, and transport-error states. | `--color-dashboard-*`, `OperationsDashboard` |
| `Ontology/OccupationRatingProfile` | Select an imported release/source, enter an exact O*NET-SOC code, inspect the published value beside its sample/error and warning, then open the rating or scale artifact. `InteractiveEvidenceReady`, `EvidenceReady`, `NarrowViewport`, `CatalogEmpty`, `CatalogUnavailable`, `SourceUnavailable`, and `EmptyOccupation` cover the catalog-backed form, populated table, horizontal mobile access, and honest catalog/profile absence states. | `OccupationRatingProfile`, native select/table, `--color-border`, `--size-control-min` |
| `Ontology/OccupationRatingProfile` | Select an imported release/source, then a published occupation that has observations in that source, inspect the published value beside its sample/error and warning, then open the rating or scale artifact. `InteractiveEvidenceReady`, `EvidenceReady`, `NarrowViewport`, `CatalogEmpty`, `CatalogUnavailable`, `OccupationCatalogEmpty`, `OccupationCatalogUnavailable`, `SourceUnavailable`, and `EmptyOccupation` cover the catalog-backed form, populated table, horizontal mobile access, and honest catalog/profile absence states. | `OccupationRatingProfile`, native select/table, `--color-border`, `--size-control-min` |
| `Post/SimilarVocPanel` | Compare ontology/semantic similar VOC and prior action evidence, then open the source; unavailable states show no fabricated TEPP theta or weight. | `SimilarVocPanel.css`, `SimilarVocPanel` |
| `Evidence/CitationChip` | Click a cited title to open that source post. | `--color-chip-border`, `--radius-chip`, `CitationChip` |
| `Evidence/OrganizationAliasChip` | Click a cataloged org; the parenthetical is the unique corroborated SKOS companion. | `--color-chip-border`, `--radius-chip`, `OrganizationAliasChip` |
Expand Down
2 changes: 1 addition & 1 deletion frontend/src/App.css
Original file line number Diff line number Diff line change
Expand Up @@ -1448,7 +1448,7 @@

.occupation-rating-form {
display: grid;
grid-template-columns: minmax(14rem, 1fr) minmax(16rem, 1fr) auto;
grid-template-columns: minmax(16rem, 1fr) minmax(18rem, 1.4fr) auto;
align-items: end;
gap: var(--space-control-gap);
margin: 1rem 0;
Expand Down
Loading