feat: Phase 1 product schema (3NF PostgreSQL, real DB tests)

ARCHITECTURE.md

@@ -98,3 +98,20 @@ flowchart LR
 
 See [`docs/lineage-bi-research-notes.md`](docs/lineage-bi-research-notes.md)
 for the full APA 7th reference list this design is grounded in.
+
+## Product schema (Phase 1 of a larger roadmap)
+
+`lineageweave`'s reconstruction pipeline above is being wrapped in a real
+product: corp/PU-code accounts, ABAC/RBAC, posts, Keyman extraction, a
+Knowledge Graph, corporate hierarchy, and issue tickets. See
+[`docs/adr/0001-demo-identity-and-data-boundary.md`](docs/adr/0001-demo-identity-and-data-boundary.md)
+for the identity/data scope decision (real infrastructure, synthetic
+identities and content) and `migrations/0001_initial_schema.sql` for the
+3NF PostgreSQL schema (`common_lookup_value`, `corporate_entity`,
+`process_unit`, `user_account`, `account_affiliation`, `access_role` /
+`role_permission` / `account_role_assignment`, `abac_policy`, `post` /
+`post_counterparty_entity`, `person` / `person_affiliation` /
+`post_person_mention`, `knowledge_graph_edge`, `issue_ticket`,
+`post_lineage_edge`). Real-database tests: `tests/test_schema.py`
+(skipped without a reachable PostgreSQL server, same pattern as the
+real-provider LLM tests).

CHANGELOG.md

@@ -4,6 +4,36 @@ All notable changes to this project are documented here. Format follows
 [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); versioning follows
 [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.4.0] - 2026-08-13
+
+### Added
+
+- Milestone 4, Phase 1 begins: LineageWeave's product schema.
+  `migrations/0001_initial_schema.sql` -- a 3NF PostgreSQL schema
+  (snake_case, 2+ word object names) covering accounts (corp/PU code as
+  attributes, not the login key), a shared `common_lookup_value` ENUM
+  table, posts + visibility, ABAC/RBAC, VOC-type and entity-relationship
+  classification, Keyman (`cataloged_person` + N:N `person_affiliation`), a
+  `knowledge_graph_edge` table, `issue_ticket`, and a self-referencing
+  `corporate_hierarchy` via `corporate_entity.parent_entity_id`.
+- `docs/adr/0001-demo-identity-and-data-boundary.md`: the identity/data
+  scope decision for this expansion -- real infrastructure (Postgres,
+  Valkey, a real OIDC provider), synthetic identities and content, because
+  Keyman extraction catalogs real named individuals (including
+  non-consenting external counterparties) and a real production identity
+  provider would re-identify the source organization through account/data
+  structure even with zero literal company-name strings in source files.
+- `tests/test_schema.py`: real-database tests (skipped without a
+  reachable PostgreSQL server) proving the migration applies cleanly, a
+  multi-level corporate-hierarchy recursive query returns the right
+  shape, and an invalid lookup code is genuinely rejected by a foreign
+  key -- caught and fixed a real bug in the process (an accidental
+  `deferrable initially deferred` on one FK silently weakened its
+  integrity check within a transaction).
+- New citations staged for Phase 2/3: Tong et al. (2006, random walk with
+  restart -- Knowledge Graph per-node traversal depth) and Bhattacharya &
+  Getoor (2007, collective entity resolution -- corporate hierarchy).
+
 ## [0.3.0] - 2026-08-13
 
 ### Fixed

docs/adr/0001-demo-identity-and-data-boundary.md

@@ -0,0 +1,90 @@
+# ADR 0001 — Demo-grade identity and data boundary for LineageWeave's product features
+
+**Decision status:** Accepted
+**Date:** 2026-08-13
+
+## Context
+
+LineageWeave is growing from a synthetic-data DAG-reconstruction prototype
+into a fuller product surface: corporate/PU-code login, ABAC/RBAC, a
+Keyman (key-contact) extraction and Knowledge Graph feature, a corporate
+hierarchy tree, and a real PostgreSQL-backed React frontend, wired to a
+real identity provider (Keyverse) and a real event queue (Valkey).
+
+Two of these features are qualitatively different from anything built so
+far in this repository:
+
+1. **Keyman extraction** derives and persists the *names, roles, and
+   inter-organizational relationships of real individuals* -- including
+   people at counterparty organizations who never consented to being
+   catalogued in a Knowledge Graph -- from free-text record content.
+2. **Keyverse login** would wire this repository to the organization's
+   real production identity provider and real employee accounts.
+
+Every prior milestone in this repository kept a strict boundary: real
+data analysis happens locally only, is never committed, and the public
+repository ships synthetic data and pluggable clients only. A working
+product feature that logs in with real accounts and displays real
+extracted people is a different shape of risk than an optionally-real-
+provider-gated test -- it is the shipped, running behavior of a public,
+company-name-scrubbed repository, not an opt-in test path.
+
+## Decision
+
+LineageWeave's product features are built and demonstrated against a
+**real, working, but synthetic-identity stack**, not the organization's
+real production tenant or real record content:
+
+1. **Identity**: a real OIDC-compatible auth flow, stood up via Docker
+   Compose (not a "recorded HTTP-shaped adapter" stub -- the login flow
+   genuinely works end to end), seeded with synthetic demo accounts,
+   corporate codes, and PU codes. Not the organization's real Keyverse
+   tenant.
+2. **Content**: Keyman extraction, Knowledge Graph traversal, corporate
+   hierarchy AI, and lineage popups are demonstrated against synthetic
+   fixture records (fabricated organizations, fabricated people) --
+   never real extracted third-party names or real record content.
+3. **Real data analysis** (embeddings, LLM adjudication, chunking, image
+   OCR against the actual real dataset) continues exactly as established
+   in milestones 1-3: runs locally only, results never committed to any
+   repository, and only aggregate/non-identifying findings are ever
+   referenced anywhere else. This ADR does not change that boundary --
+   it clarifies that the *product's shipped, running behavior* is a
+   separate concern from the *private local analysis*, and only the
+   latter touches real content.
+
+## Rationale
+
+- Cataloguing real, named individuals -- especially non-consenting
+  external counterparties -- into a persistent, queryable graph is a
+  substantial privacy/consent question a development-process instruction
+  cannot resolve on those individuals' behalf. Building the *mechanism*
+  against synthetic people demonstrates the same capability without that
+  question being open.
+- A real login flow wired to the organization's actual identity
+  provider and real employee accounts, inside a repository whose whole
+  point is to never identify the source organization in its files, would
+  re-introduce that identification through a different channel (account
+  structure, real corporate/PU codes) even with zero literal company-name
+  strings in the code.
+- This still fully satisfies "no recorded/mocked adapters" -- the auth
+  flow, the database, the event queue, and the frontend are all real and
+  working; only the *identities and content* flowing through them are
+  synthetic, exactly like `lineageweave/fixtures.py` already establishes
+  for the DAG-reconstruction pipeline.
+
+## Consequences
+
+- Docker Compose must stand up a genuinely functional OIDC provider (not
+  a stub), PostgreSQL, and Valkey -- "it works with fake data" is the bar,
+  not "it works."
+- If a real deployment against the organization's actual Keyverse tenant
+  and real record content is wanted later, that is a distinct, explicit
+  decision for a private, non-public deployment -- not something this
+  public repository's default configuration does.
+
+## Related
+
+Builds on the data-handling discipline established across this
+repository's prior milestones (see `docs/lineage-bi-research-notes.md`
+and `AGENTS.md`'s "no real data, ever" rule).

docs/lineage-bi-research-notes.md

@@ -213,6 +213,8 @@ Crispin, M., & Murchison, K. (2008). *Internet Message Access Protocol (IMAP) -
 
 Doddington, G., Mitchell, A., Przybocki, M., Ramshaw, L., Strassel, S., & Weischedel, R. (2004). The Automatic Content Extraction (ACE) program -- Tasks, data, and evaluation. In *Proceedings of the Fourth International Conference on Language Resources and Evaluation (LREC 2004)* (pp. 837-840). European Language Resources Association.
 
+Bhattacharya, I., & Getoor, L. (2007). Collective entity resolution in relational data. *ACM Transactions on Knowledge Discovery from Data*, *1*(1), 5-es. https://doi.org/10.1145/1217299.1217304
+
 Fellegi, I. P., & Sunter, A. B. (1969). A theory for record linkage. *Journal of the American Statistical Association*, *64*(328), 1183-1210. https://doi.org/10.2307/2286061
 
 Hearst, M. A. (1997). TextTiling: Segmenting text into multi-paragraph subtopic passages. *Computational Linguistics*, *23*(1), 33-64.
@@ -225,10 +227,29 @@ Raudenbush, S. W., & Bryk, A. S. (2002). *Hierarchical linear models: Applicatio
 
 Resnick, P. (2008). *Internet Message Format* (RFC 5322). IETF. https://doi.org/10.17487/RFC5322
 
+Tong, H., Faloutsos, C., & Pan, J.-Y. (2006). Fast random walk with restart and its applications. *Proceedings of the Sixth International Conference on Data Mining (ICDM'06)*, 613-622. https://doi.org/10.1109/ICDM.2006.70
+
 WHATWG. (2026). *HTML Living Standard — sections 4.3 (sectioning content) and 4.4 (grouping content)*. https://html.spec.whatwg.org/
 
 Zawinski, J. (1997). *Message threading* [Design note]. jwz.org. https://www.jwz.org/doc/threading.html
 
 Additional context on the Fugu / Conductor / TRINITY test-time-compute-allocation research the `llm` channel's design follows is maintained in
 [contextual-orchestrator's own literature register](https://github.com/ContextualWisdomLab/contextual-orchestrator/blob/main/docs/architecture.md)
 rather than duplicated here, so the two repos do not drift out of sync.
+
+## Staged for later phases (cited now so they are not lost)
+
+Two citations above ground work not yet implemented, staged for the
+product roadmap in `docs/adr/0001-demo-identity-and-data-boundary.md`:
+
+- **Tong et al. (2006)** -- random walk with restart -- backs the
+  Knowledge Graph traversal depth question (each Keyman/company/post node
+  in the graph can warrant a different effective traversal depth; RWR's
+  restart probability gives a continuous, per-node relevance weighting
+  rather than a single fixed hop count, which is the shape the product
+  requirement describes).
+- **Bhattacharya & Getoor (2007)** -- collective entity resolution --
+  backs the corporate-hierarchy-tree feature (resolving "Acme Group" /
+  "Acme Electronics Korea" / "Acme Electronics Gwangju Plant" as related
+  entities in one collective inference pass rather than independent
+  string-matching per pair).

lineageweave/__init__.py

@@ -12,4 +12,4 @@
 
 __all__ = ["Edge", "Record", "Tree", "reconstruct"]
 
-__version__ = "0.3.0"
+__version__ = "0.4.0"

lineageweave/image_content.py

@@ -101,7 +101,14 @@ class ImageContentClient(Protocol):
 
     available: bool
 
-    def describe(self, image_bytes: bytes, mime_type: str) -> ImageDescription: ...
+    def describe(self, image_bytes: bytes, mime_type: str) -> ImageDescription:
+        """Return OCR text, caption, and tags for one image.
+
+        Implementations must raise if they cannot produce a description.
+        Protocol stubs raise ``NotImplementedError`` so a no-op body is
+        never treated as a successful empty result.
+        """
+        raise NotImplementedError
 
 
 class NullImageContentClient: