Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
84 commits
Select commit Hold shift + click to select a range
30f604f
feat: add bounded ontology and provenance explorer
seonghobae Aug 21, 2026
5b2fc2e
fix: keep ontology JSON-LD exports filtered
seonghobae Aug 21, 2026
2e7ed08
fix: validate ontology neighborhood request bounds
seonghobae Aug 21, 2026
bf588b7
fix: escape ontology CSV formulas
seonghobae Aug 21, 2026
c5babaa
fix: make ontology layout ordering deterministic
seonghobae Aug 21, 2026
fe82d1f
fix: make ontology explorer controls accessible
seonghobae Aug 21, 2026
d8971b2
fix: translate ontology post node labels
seonghobae Aug 21, 2026
f83269a
fix: deduplicate ontology node evidence
seonghobae Aug 21, 2026
b369d30
fix: drop unlabeled ontology edges
seonghobae Aug 21, 2026
e0b3bd2
fix: harden typed ontology neighborhood boundaries
seonghobae Aug 21, 2026
977e6b2
Merge remote-tracking branch 'origin/feat/ontology-provenance-explore…
seonghobae Aug 21, 2026
2edf3d0
fix: bound ontology fact loading by requested edge limit
seonghobae Aug 21, 2026
bca21ce
fix: complete ontology explorer verification coverage
seonghobae Aug 21, 2026
1338e66
fix: hide unauthorized ontology parents
seonghobae Aug 21, 2026
a4fe025
Merge remote-tracking branch 'origin/feat/ontology-provenance-explore…
seonghobae Aug 21, 2026
bf76b4a
Merge remote-tracking branch 'origin/feat/ontology-provenance-explore…
seonghobae Aug 21, 2026
7134626
Merge remote-tracking branch 'origin/feat/ontology-provenance-explore…
seonghobae Aug 21, 2026
9f70d71
Merge remote-tracking branch 'origin/feat/ontology-provenance-explore…
seonghobae Aug 21, 2026
4ab2970
test: reproduce missing Storybook viewport addon
seonghobae Aug 21, 2026
cddf0dd
fix: declare Storybook viewport addon
seonghobae Aug 21, 2026
6adb039
fix: register Storybook viewport addon
seonghobae Aug 21, 2026
33be289
fix: use Storybook 10 built-in viewport feature
seonghobae Aug 21, 2026
551c1c5
fix: keep Storybook 10 essentials zero-config
seonghobae Aug 21, 2026
82efc9f
fix: declare Storybook built-in viewport options
seonghobae Aug 21, 2026
e06d85f
test: lock Storybook 10 built-in viewport contract
seonghobae Aug 21, 2026
770403e
fix: bind narrow ontology story to mobile viewport
seonghobae Aug 21, 2026
54ae7ab
test: reproduce cutoff and uppercase focus defects
seonghobae Aug 21, 2026
de927dd
fix: apply ontology cutoff before SQL limit
seonghobae Aug 21, 2026
95950af
test: require localized ontology stabilization copy
seonghobae Aug 21, 2026
735447d
feat: localize ontology paging and provenance guidance
seonghobae Aug 21, 2026
c59ca52
test: reproduce ontology paging and provenance UX defects
seonghobae Aug 21, 2026
0789fe8
fix: stabilize ontology paging and provenance UX
seonghobae Aug 21, 2026
fba2220
test: reproduce bounded window and disconnected first page
seonghobae Aug 21, 2026
dd48881
fix: preserve proximity and source truncation truth
seonghobae Aug 21, 2026
a76d4c4
fix: expose bounded ontology source windows
seonghobae Aug 21, 2026
20ac2b1
fix: keep ontology loader test doubles compatible
seonghobae Aug 21, 2026
fadb146
test: reproduce duplicate ontology focus rule
seonghobae Aug 21, 2026
34e6643
test: reproduce endpoint N+1 and shallow evidence expansion
seonghobae Aug 21, 2026
ffcc7c6
test: require a next action for hard ontology bounds
seonghobae Aug 21, 2026
bfe5d18
test: require localized hard-bound guidance
seonghobae Aug 21, 2026
0c85ce3
feat: localize hard ontology query-bound guidance
seonghobae Aug 21, 2026
e6e36ac
fix: validate provider chat response envelopes
seonghobae Aug 21, 2026
a9369a0
fix: close remaining provider error leaks
seonghobae Aug 21, 2026
a8d4d43
fix: normalize summary enrichment failures
seonghobae Aug 21, 2026
7fa92e3
fix: normalize unexpected provider failures
seonghobae Aug 21, 2026
dbb96e9
fix: keep TEPP and worker errors provider-safe
seonghobae Aug 21, 2026
961a827
fix: bound and batch ontology neighborhood evidence
seonghobae Aug 21, 2026
ac730f6
fix: preserve ontology visibility query compatibility
seonghobae Aug 21, 2026
4b9d4d4
fix: restore ontology explorer next actions
seonghobae Aug 21, 2026
a1e4fdd
fix: honor ontology cutoff and canonical focus UUIDs
seonghobae Aug 21, 2026
67dfcf2
Merge branch 'feat/ontology-provenance-explorer-v2130' of https://git…
seonghobae Aug 21, 2026
cf414d9
fix: read ontology visibility records safely
seonghobae Aug 21, 2026
8ba2021
fix: preserve provider parse failures and truncation
seonghobae Aug 21, 2026
799075d
fix: preserve ontology explorer session and parent visibility
seonghobae Aug 21, 2026
e776550
fix: authorize final ontology endpoints
seonghobae Aug 21, 2026
5b7fed2
fix: preserve ontology evidence and stale diagnostics
seonghobae Aug 21, 2026
1cb9f70
fix: make tenant settings migration replayable
seonghobae Aug 21, 2026
eb84dea
fix: harden ontology explorer paging and login return
seonghobae Aug 21, 2026
e88f386
docs: sanitize baseline and record ontology gates
seonghobae Aug 21, 2026
539b652
fix: align provider changelog release
seonghobae Aug 21, 2026
2eb6d13
test: assert ontology fact query limit argument
seonghobae Aug 21, 2026
6e591f4
feat: continue ontology neighborhoods with an opaque source cursor (v…
seonghobae Aug 21, 2026
feb55d0
Fix ontology cursor after derived edge overflow
seonghobae Aug 21, 2026
df12509
fix(ontology): remove unused subclass IRI constant
seonghobae Aug 21, 2026
129d505
fix(frontend): preserve ontology pages on continuation failure
seonghobae Aug 21, 2026
2d18879
test(frontend): retain ontology selection after page failure
seonghobae Aug 21, 2026
dc4bd02
test(frontend): align rejected ontology fixture rows
seonghobae Aug 21, 2026
13ad75e
fix: make ontology page retries effective
seonghobae Aug 21, 2026
979b429
fix(security): replace custom source cursor cipher
seonghobae Aug 21, 2026
13e576e
fix(security): keep aggregate SQL immutable
seonghobae Aug 21, 2026
202194a
test(security): preserve reviewed SQL contract
seonghobae Aug 21, 2026
bfb3760
fix(ontology): keep paging safe and stable
seonghobae Aug 21, 2026
e31047f
fix: conceal ontology error details
seonghobae Aug 21, 2026
a6af452
fix: distinguish full ontology page from truncation
seonghobae Aug 21, 2026
499e81e
test: use one ontology ingestion import style
seonghobae Aug 21, 2026
40286c1
fix(ontology-neighborhood): trim by BFS distance, not key string
seonghobae Aug 23, 2026
e11b03e
fix: allocate migration 0175 for ontology truth
seonghobae Aug 23, 2026
bef4a85
fix: restore ontology snapshot on reset
seonghobae Aug 23, 2026
5270bd5
Merge remote-tracking branch 'origin/main' into HEAD
seonghobae Aug 24, 2026
d59d667
fix: resolve silent ADR-0119 number collision from main merge
seonghobae Aug 24, 2026
dd4b3b8
fix: document the source-window page sort key closure
seonghobae Aug 24, 2026
f6ea738
Merge remote-tracking branch 'origin/main' into HEAD
seonghobae Aug 24, 2026
feb39df
merge: resolve conflicts with main (post PR #519 leftover-map axis sh…
seonghobae Aug 24, 2026
58342f2
merge: resolve conflicts with main (wave 2b, post PR #572 Valkey queue)
seonghobae Aug 24, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -46,4 +46,10 @@ LLM_GATEWAY_EMBEDDING_MODEL=
LLM_API_GATEWAY=
LLM_API_KEY=
CALDAV_BASE_URL=
RANKWEAVE_DISABLED=

# Optional process HMAC for GET /api/ontology/neighborhood source continuation.
# Empty keeps truncated-without-cursor. Do not reuse OIDC or orchestrator secrets.
# Must be at least 32 bytes when source paging is enabled.
Comment thread
devin-ai-integration[bot] marked this conversation as resolved.
ONTOLOGY_SOURCE_CURSOR_SECRET=
CONTEXTUAL_ORCHESTRATOR_ALLOWED_PROVIDER_HOSTS=
4 changes: 3 additions & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,9 @@ scattered short records. See [ARCHITECTURE.md](ARCHITECTURE.md) for the
design, [ADR 0084](docs/adr/0084-lineage-research-grounding.md) for the
normative research-grounding policy, and
[`docs/lineage-bi-research-notes.md`](docs/lineage-bi-research-notes.md) for
supporting literature and aggregate evidence.
supporting literature and aggregate evidence. Event Lineage (reconstructed
post-to-post parents) is distinct from the typed ontology neighborhood
(ADR 0168); source-window continuation is ADR 0124. Do not mix those graphs.

## Hard rule: no real data in repository artifacts

Expand Down
2 changes: 2 additions & 0 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,8 @@ flowchart LR
| `post_chat.py` | Pluggable in-popup chat's reason-and-cite step (retrieve step lives in `backend/app/post_chat_ingestion.py`) |
| `commitment_extraction.py` | Pluggable LLM derivation of a customer commitment (promise + deadline) from a post; `Null` default, `ContextualOrchestrator` real impl |
| `ontology.py` | Loads `docs/ontology/lineageweave-kg.ttl`, the formal OWL 2/RDFS/SKOS vocabulary for the Knowledge Graph's node/edge types (ADR 0004) |
| `ontology_neighborhood.py` | Bounded typed ontology/provenance neighborhood (ADR 0168); PostgreSQL stays authoritative, OWL subclass is not an instance edge |
| `ontology_source_cursor.py` | Opaque HMAC source-window continuation (ADR 0124); keyset pagination, never OFFSET |
| `period_report.py` | Fit GRM/GPCM on persisted IRT rows, FIPC-select, EAP-score a period (ADR 0003 slice 3; Bock & Mislevy, 1982) |
| `fixtures.py` | Synthetic demo dataset -- no real data ships in this repo |
| `server.py` | Legacy stdlib HTTP server for the library-level synthetic fixture demo; production uses FastAPI/PostgreSQL |
Expand Down
17 changes: 17 additions & 0 deletions CHANGELOG.d/2.13.0-ontology-explorer.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# 2.13.0 — Ontology provenance explorer

- Added `GET /api/ontology/neighborhood` as a bounded typed instance graph
(ADR 0168 / issue #341), distinct from Event Lineage.
- PostgreSQL stays authoritative; OWL/RDF/JSON-LD is a projection.
- SKOS broader comes from `corporate_entity.parent_entity_id`; OWL subclass
is rejected as an instance edge.
- Hidden endpoints remove the edge with no omitted-count side channel.
- Catalog metadata is the only source for node truth and timestamps; missing
labels and unauthorized endpoints are omitted without leaking counts.
- Neighborhood SQL now bounds traversal from the focus node, and CSV/JSON-LD
exports remain aligned with the active Buyer search filter.
- Workspace surface: Keyman **Inspect ontology neighborhood**, exact-value table,
CSV/JSON-LD export, and Storybook states. No second GNB destination.
- Session-token removal clears a previously loaded neighborhood. Static catalog
snapshots hide live refocus. Corporate parents require their own visible-post
evidence. OWL-Time is cited as a W3C Candidate Recommendation Draft.
4 changes: 4 additions & 0 deletions CHANGELOG.d/2.13.0-provider-error-boundary.md
Comment thread
seonghobae marked this conversation as resolved.
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
## Fixed

- Keep contextual-orchestrator, OIDC, RankWeave, TEPP, and durable-ingestion diagnostics behind stable product error boundaries while retaining the original exception for server-side chaining.
- Keep browser 5xx and transport failures behind the same stable client error boundary.
10 changes: 10 additions & 0 deletions CHANGELOG.d/2.14.0-ontology-source-cursor.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# 2.14.0 — Ontology source-window continuation

- Added a versioned opaque `src.v2.` AES-GCM cursor so buyers can continue an
authorized ontology neighborhood beyond the bounded SQL source window
(ADR 0124/0125 / issue #363); the custom v1 keystream format is rejected.
- Continuation uses keyset pagination of the recursive candidate window, not
`OFFSET`. Tamper, scope, version, expiry, and snapshot drift fail closed.
- `ONTOLOGY_SOURCE_CURSOR_SECRET` is a process secret, never an OIDC or
orchestrator credential. Missing or short secrets withhold the cursor.
- The explorer accumulates subsequent pages without losing selected evidence.
18 changes: 18 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,13 @@ All notable changes to this project are documented here. Format follows

### Added

- Ontology neighborhoods can continue beyond the bounded SQL source window
with a versioned opaque HMAC cursor and keyset pagination (ADR 0124 / #363).
A missing process secret keeps the truncated-without-cursor contract.
- Bounded ontology/provenance neighborhood (`GET /api/ontology/neighborhood`)
with typed Post/Person/CorporateEntity/Team nodes, SKOS broader distinct
from OWL subclass, truth-status vocabulary, knowledge-cutoff binding, and
a Keyman-panel explorer that is not Event Lineage (ADR 0168 / #341).
- Corroborated SKOS `altLabel` / `prefLabel` pairs now expand corporate
catalog candidates so a synthetic short form (`AGP`) and full form
(`Aurora Grid Power`) bind one `corporate_entity` row instead of
Expand Down Expand Up @@ -48,6 +55,17 @@ All notable changes to this project are documented here. Format follows
- `make smoke` and `make seed` now run through the locked project `uv`
environment, so local OIDC and synthetic-data workflows resolve the same
pinned dependencies as CI.
- Ontology neighborhoods now enforce request bounds before database access,
apply node-level ABAC, omit unlabeled endpoints, preserve catalog-owned node
metadata, and keep typed endpoint IDs unambiguous. Workspace CSV and JSON-LD
exports now represent the same filtered graph.
- All OpenAI-compatible chat-completion consumers now validate the shared
response envelope before parsing it, preventing malformed provider bodies
from escaping as raw `KeyError` or response-shape details.
- Ontology Explorer now clears a previously loaded neighborhood when the
session token is removed, hides live refocus on static catalog snapshots,
and looks up corporate-parent visibility independently of the child. OWL-Time
is cited as a W3C Candidate Recommendation Draft.

## [2.12.26] - 2026-08-24

Expand Down
20 changes: 11 additions & 9 deletions backend/app/analysis_run_ingestion.py
Original file line number Diff line number Diff line change
Expand Up @@ -148,6 +148,15 @@
order by run.requested_at desc
"""

_COUNTS_BY_RUN_SQL = """
select run.analysis_run_id, counts.count_type_code, counts.count_value
from analysis_run run
join analysis_source_count counts
on counts.analysis_source_snapshot_id = run.analysis_source_snapshot_id
where run.analysis_run_id = any($1::uuid[])
order by counts.count_type_code
"""


def scope_grouping_key(row: Any) -> str | None:
"""Persist the reconstruct grouping key for the run's authorized scope.
Expand Down Expand Up @@ -195,16 +204,9 @@ async def _counts_by_run(
"""Load aggregate snapshot counts for the given runs."""
if not run_ids:
return {}
# Safe SQL: this immutable aggregate query has closed schema text; run ids are bound below.
# Safe SQL: this immutable aggregate query has closed schema text; run ids remain bound below.
rows = await conn.fetch( # nosemgrep: python.lang.security.audit.sqli.asyncpg-sqli.asyncpg-sqli
f"""
select run.analysis_run_id, counts.count_type_code, counts.count_value
from analysis_run run
join analysis_source_count counts
on counts.analysis_source_snapshot_id = run.analysis_source_snapshot_id
where run.analysis_run_id = any($1::uuid[])
order by counts.count_type_code
""",
_COUNTS_BY_RUN_SQL,
run_ids,
)
grouped: dict[str, list[asyncpg.Record]] = {}
Expand Down
2 changes: 2 additions & 0 deletions backend/app/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ class Settings:
tepp_api_key: str
caldav_base_url: str
rankweave_disabled: bool
ontology_source_cursor_secret: str

@property
def keycloak_jwks_uri(self) -> str:
Expand Down Expand Up @@ -175,4 +176,5 @@ def load_settings() -> Settings:
.strip()
.lower()
in {"1", "true", "yes", "on"},
ontology_source_cursor_secret=os.environ.get("ONTOLOGY_SOURCE_CURSOR_SECRET", "").strip(),
)
16 changes: 15 additions & 1 deletion backend/app/global_ask_queue.py
Original file line number Diff line number Diff line change
Expand Up @@ -280,7 +280,21 @@ async def process_global_ask_job(
# the task silently and strand the row `running` until orphan
# recovery (observed live) — every failure settles the job.
_logger.exception("global ask job failed for job_id=%s", job_id)
detail = str(exc) or f"job exceeded the {JOB_DEADLINE_SECONDS}s deadline"
if isinstance(exc, (PermissionError, ConnectionError)):
# Raised locally with a pre-authored, safe message (permission
# state / missing config) — never a provider-boundary leak.
detail = str(exc)
elif isinstance(exc, asyncio.TimeoutError):
detail = f"job exceeded the {JOB_DEADLINE_SECONDS}s deadline"
else:
# Provider responses/exceptions can carry credentials, gateway
# diagnostics, or model output (ADR 0123): never persist the
# raw exception text as a durable `failure_detail`. The
# traceback just logged keeps it for operator debugging only.
detail = (
"Ask Agent is unavailable: contextual-orchestrator returned "
"no complete evidence object"
)
Comment on lines +283 to +297

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 ConnectionError branch keeps raw exception text

The settlement handler keeps str(exc) for ConnectionError (global_ask_queue.py). A socket-level ConnectionError can carry an internal host/address that becomes a durable failure_detail, weakening the ADR 0123 leak goal. Provider calls flow through http_client (raising HttpClientError/OSError), so no current path reaches this branch with provider detail, but the tuple is broad.

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

async with pool.acquire() as conn:
await conn.execute(
"""
Expand Down
Loading
Loading