Skip to content
Draft
Show file tree
Hide file tree
Changes from 27 commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
23e8acf
test(consent): require atomic consent outbox persistence
seonghobae Aug 14, 2026
0b4dae6
feat(consent): persist lifecycle with bound outbox event
seonghobae Aug 14, 2026
15f9842
feat(consent): export transactional propagation adapter
seonghobae Aug 14, 2026
d499867
test(consent): cover propagation envelope boundaries
seonghobae Aug 14, 2026
9c65cb2
test(consent): preserve consent failure boundary in outbox composition
seonghobae Aug 14, 2026
bf1dfe7
style(consent): apply rustfmt to propagation tests
seonghobae Aug 14, 2026
f1d23ff
style(consent): apply rustfmt to outbox transaction tests
seonghobae Aug 14, 2026
0c414a6
style(consent): terminate propagation module with newline
seonghobae Aug 14, 2026
1ff6411
style(consent): terminate propagation transaction test
seonghobae Aug 14, 2026
43a7ddb
test(consent): reject stale propagation after revocation
seonghobae Aug 14, 2026
65ee847
fix(consent): bind propagation to latest consent change
seonghobae Aug 14, 2026
bf1a8d5
test(consent): reject cross-tenant propagation envelopes
seonghobae Aug 15, 2026
f2c050e
fix(consent): bind propagation to authorized tenant
seonghobae Aug 15, 2026
1f71fbb
test(consent): carry tenant authority into error boundary
seonghobae Aug 15, 2026
2d4da10
test(consent): bind latest-event propagation to tenant
seonghobae Aug 15, 2026
3744334
fix(consent): reject tenant alias spellings
seonghobae Aug 15, 2026
93092fd
style(consent): apply rustfmt to latest-event test
seonghobae Aug 15, 2026
cf78b37
style(consent): apply rustfmt to transaction test
seonghobae Aug 15, 2026
3180620
Merge branch 'main' into feat/consent-outbox-transaction-20260814-1821
opencode-agent[bot] Aug 16, 2026
4ebc02f
fix(consent): bind outbox to durable ledger tail
cursoragent Aug 16, 2026
040bcf7
test(consent): cover durable-tail fail-closed branches
cursoragent Aug 16, 2026
ee62f08
fix(consent): order durable tail by insertion time
cursoragent Aug 16, 2026
3f72446
docs(consent): name Active PR #120 as the outbox merge candidate
cursoragent Aug 16, 2026
d5b2c93
fix(consent): fail closed on ambiguous durable tail
cursoragent Aug 16, 2026
729a2bd
docs(consent): name Active PR #134 as the outbox merge candidate
cursoragent Aug 16, 2026
896015f
fix(consent): stop documenting event-identity tail order
cursoragent Aug 16, 2026
75d71eb
docs(consent): name Active PR #142 as the outbox merge candidate
cursoragent Aug 16, 2026
47568be
test(recovery): seed processing claim_deadline_at after inbox expiry
seonghobae Aug 16, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ All notable product and architecture changes are recorded here. Releases use imm
- PostgreSQL persistence for immutable version-pinned scoring-request identity, with exact replay and fail-closed rebinding of snapshot or measurement-version evidence.

- PostgreSQL 18 purpose-specific consent persistence for one participant-bound ledger plus append-only grant/revoke events, with exact replay, fail-closed conflicting event identity, independent research-scope shape, and `READ COMMITTED` replay classification.
- Atomic consent-change persistence with a tenant-, participant-, and insertion-order durable-ledger-tail-bound outbox event in the same caller-owned transaction. The durable event set must appear in the submitted ledger, and an ambiguous physical tail (shared `occurred_at_unix_ms` and `created_at`) fails closed instead of using opaque event identity as a correctness key, so a later stored revocation cannot be committed beside stale grant propagation.
- PostgreSQL 18 inbox-consumption persistence so an accepted inbox receipt is not side-effect completion: pending identity, fenced processing claims that cannot be stolen, expire-and-reclaim of a crashed claim without transferring the fence, local or claimed completion, and poison quarantine, with exact replay and `READ COMMITTED` fail-closed classification.
- Participant-owned consent authorization: `ResourceKind::ConsentLedger` and `ProductPermission::ManageOwnConsent` reuse the fail-closed tenant, resource-kind, and owner checks so consent cannot be treated as an untyped generic concern.
- PostgreSQL expired-lease recovery for scoring jobs: an expired leased row becomes a due retry or quarantine with typed `lease_expired` evidence, and a later claim issues the next fencing token instead of inheriting the crashed worker's lease.
Expand Down
6 changes: 4 additions & 2 deletions docs/TRACEABILITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ An active PR, architecture document, conversation decision, or scheduler plan is
| Instrument publication requires intended-use scientific/right/locale evidence | PRD §6, §9, §10 | Measurement Governance; publication evidence gate | ADR-0004, ADR-0013, ADR-0019 | **Implemented** policy gate and immutable evidence provenance in `src/instrument.rs`; each real instrument still requires its own rights/locale/scientific evidence artifacts before publication |
| Optional Keyverse account linking | PRD §3.1, §9.7 | TRD §10; UML identity-link lifecycle | ADR-0003, ADR-0020 | **Partially implemented**: issuer-scoped first-link fail-closed domain primitive in `src/participant.rs`; append-only unlink/relink/recovery history, persistence, audit, and transport remain Target |
| Cross-cutting tenant/task authorization | PRD §7, §9 | TRD §11; Security/Data | ADR-0001, ADR-0003 | **Implemented** fail-closed domain gate in `src/authorization.rs` binds consent operations to participant-owned `ConsentLedger` / `ManageOwnConsent`; persistence/policy-adapter/public-transport integration remains Target |
| Purpose-specific consent | PRD §5, §9.6 | TRD §12 | ADR-0006 | **Implemented** domain contract in `src/consent.rs` plus `migrations/0005_consent_lifecycle.sql` / `src/postgres_consent.rs` purpose-specific ledgers; HTTP transport remains Target |
| Purpose-specific consent | PRD §5, §9.6 | TRD §12 | ADR-0006 | **Implemented** domain contract in `src/consent.rs` plus `migrations/0005_consent_lifecycle.sql` / `src/postgres_consent.rs` purpose-specific ledgers; HTTP transport remains Target. Active PR #142 composes that ledger with a bound outbox after locking the insertion-order durable tail and failing closed on an ambiguous physical tail |
| Explicit research contribution + withdrawal | PRD §5 | TRD §12, §14–15 | ADR-0006, ADR-0007 | **Implemented** product-domain lifecycle in `src/consent.rs`; dataset snapshot/release integration is Target |
| Participant export/deletion | PRD §3.1, §9, §11 | TRD §13 | ADR-0006 | **Implemented** domain lifecycle in `src/data_rights.rs` plus `migrations/0003_data_rights_propagation.sql` and `src/postgres_data_rights.rs`; dependent-system execution remains Target |
| Research identity separation | PRD §5, §11 | TRD §14; ERD restricted linkage | ADR-0003, ADR-0006, ADR-0007, ADR-0020 | Partially implemented via research-contribution identity separation; restricted linkage persistence is Target |
Expand Down Expand Up @@ -75,7 +75,7 @@ An active PR, architecture document, conversation decision, or scheduler plan is
| No cross-service DB access | TRD §1–2; ADR-0015 | architecture policy only | deployment credential/fitness-function test |
| Initial physical persistence target is upstream PostgreSQL 18.x | ADR-0015; Deployment/Operations | **Implemented subset** in `migrations/0001_integration_delivery.sql`, `migrations/0002_scoring_job_state.sql`, `migrations/0003_data_rights_propagation.sql`, `migrations/0005_consent_lifecycle.sql`, `migrations/0006_instrument_release.sql`, `migrations/0011_scoring_request.sql`, `migrations/0012_integration_consumption.sql`, matching adapters, and PostgreSQL operational-store readiness | remaining product aggregates, crash/restart restore acceptance |
| No default tenant for writes | TRD §11; Security/Data | authorization-domain primitive exists; persistence remains Target | persistence/API tenant negative tests |
| Tenant-bound transactional outbox/inbox | TRD §19–20; ADR-0014/0015 | `src/integration.rs` domain envelope/inbox/retry contracts plus PostgreSQL tenant/source-scoped integration evidence, delivery-attempt persistence, and inbox consumption | durable side-effect processing completion, poison-message/crash recovery, broader aggregate transaction integration |
| Tenant-bound transactional outbox/inbox | TRD §19–20; ADR-0014/0015 | `src/integration.rs` domain envelope/inbox/retry contracts plus PostgreSQL tenant/source-scoped integration evidence, delivery-attempt persistence, and inbox consumption | durable side-effect processing completion, poison-message/crash recovery, broader aggregate transaction integration. Active PR #142 binds one consent change to one outbox row after locking the insertion-order durable ledger tail and rejecting an ambiguous `(occurred_at_unix_ms, created_at)` tail |
| Inbox receipt is not side-effect completion | ADR-0014/0015; UML integration sequence | `src/integration.rs` states/retry semantics; PostgreSQL inbox consumption persists pending/processing/completed and expire-and-reclaim | live adapter crash/retry tests |
| Liveness is distinct from operation readiness | Operability §3–4; ADR-0017 | **Implemented** in `src/health.rs` and `src/postgres_health.rs`: liveness is modeled independently from operation-scoped readiness and PostgreSQL write-readiness | live transport probes, metrics, and deployment-profile acceptance |
| Optional capability outage does not fail unrelated work | Operability §3–4; ADR-0011/0017 | **Implemented** in `src/health.rs` and `src/postgres_health.rs`: readiness evaluates only capabilities required by the selected operation and maps PostgreSQL evidence onto that contract | degraded-mode transport/integration tests |
Expand Down Expand Up @@ -134,6 +134,8 @@ Still-Target logical modules/adapters include remaining product aggregate persis

**Active PR** #76 data-rights processing-start persistence is not protected-main truth until an unchanged reviewed/check-clean head is integrated. Identity-verified requests persist an immutable operation identity and processing-start time under `FOR UPDATE` so later lifecycle composition cannot race the classified row. Dependent-system execution remains outside this slice.

**Active PR** #142 consent/outbox composition is not protected-main truth until an unchanged reviewed/check-clean head is integrated. `src/postgres_consent_propagation.rs` persists one purpose-specific ledger snapshot and one `psychometrics_commons` outbox envelope in the caller-owned transaction, then locks `consent_ledger`, requires every durable `consent_event` identity to appear in the submitted ledger, and requires a unique durable tail—ordered by occurrence time, then `created_at`—to match `causation_ref` and occurrence time. An ambiguous physical tail fails closed. Do not merge #70 at `3180620`, #112 at `040bcf7`, #120 at `3f72446`, #123 at `6643041`, or #134 at `729a2bd`. HTTP consent transport and live outbox dispatch remain outside this slice.

## 5. ADR traceability by concern

| Concern | Governing ADR(s) |
Expand Down
16 changes: 15 additions & 1 deletion docs/adr/0006-consent-data-rights-and-research-separation.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,10 +64,12 @@ Construct-relevant personal data remains available to authorized workflows when
4. Public research data contains no Keyverse subject or operational participant reference.
5. Service denial cannot be based on refusal of optional research contribution.
6. Data-rights operations are tenant-scoped and identity-verified.
7. Two durable consent events that share occurrence time and physical insertion time fail closed; opaque event identity is not a correctness tie-breaker.
8. A later consent-receipt or notice interchange, when added, must map onto ISO/IEC TS 27560:2023 consent-record structure and ISO/IEC 29184:2020 notice/consent presentation. This decision does not emit those receipts yet.

## Failure behavior

If consent verification is unavailable, optional research processing fails closed while core assessment processing may continue under its valid service basis. An ambiguous withdrawal or identity conflict enters manual adjudication; it is not auto-resolved by email or display name.
If consent verification is unavailable, optional research processing fails closed while core assessment processing may continue under its valid service basis. An ambiguous withdrawal, identity conflict, or physically tied consent tail enters fail-closed rejection or manual adjudication; it is not auto-resolved by email, display name, or opaque event-reference order.

## Validation

Expand All @@ -86,3 +88,15 @@ If consent verification is unavailable, optional research processing fails close
## Reversal conditions

Revisit individual retention or withdrawal mechanics when a deployment's law or ethics approval imposes stricter requirements. The separation of service and research purposes remains mandatory.

## References

European Parliament & Council of the European Union. (2016). Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data (General Data Protection Regulation). *Official Journal of the European Union, L 119*, 1–88. https://eur-lex.europa.eu/eli/reg/2016/679/oj

International Organization for Standardization. (2020). *Information technology — Online privacy notices and consent* (ISO/IEC 29184:2020). https://www.iso.org/standard/70331.html

International Organization for Standardization. (2023). *Privacy technologies — Consent record information structure* (ISO/IEC TS 27560:2023). https://www.iso.org/standard/80392.html

National Institute of Standards and Technology. (2020). *NIST Privacy Framework: A tool for improving privacy through enterprise risk management, version 1.0* (NIST Cybersecurity White Paper). https://doi.org/10.6028/NIST.CSWP.01162020

World Medical Association. (2024). World Medical Association Declaration of Helsinki: Ethical principles for medical research involving human participants. *JAMA, 333*(1), 71–74. https://doi.org/10.1001/jama.2024.21972
8 changes: 8 additions & 0 deletions docs/adr/0015-persistence-and-transaction-boundaries.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,8 @@ An inbox row that merely proves receipt is never marked `completed` before the r

Consent decisions and data-rights lifecycle events are append-only evidence. External propagation of deletion/export/research changes is asynchronous and reconciled; local state never claims an external effect completed until evidence exists.

A consent-change write that must propagate shares the caller-owned transaction with its bound outbox row. After the submitted ledger snapshot is persisted, the same transaction locks `consent_ledger`, requires every durable `consent_event` identity to appear in the submitted ledger, and requires a unique durable tail—ordered by `occurred_at_unix_ms`, then physical `created_at`—to equal the envelope `causation_ref` and occurrence time. Equal server timestamps therefore keep a later-inserted revocation ahead of an earlier grant. Two rows that also share `created_at` fail closed; opaque `event_ref` is not a correctness tie-breaker. Callers roll back when either adapter fails so newly accepted consent evidence cannot survive without its outbox record.

## Concurrency and idempotency

Physical constraints must enforce equivalents of:
Expand Down Expand Up @@ -303,6 +305,12 @@ The physical database technology or decomposition may change if scale, residency

## References

European Parliament & Council of the European Union. (2016). Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data (General Data Protection Regulation). *Official Journal of the European Union, L 119*, 1–88. https://eur-lex.europa.eu/eli/reg/2016/679/oj

Hohpe, G., & Woolf, B. (2003). *Enterprise integration patterns: Designing, building, and deploying messaging solutions*. Addison-Wesley.

PostgreSQL Global Development Group. (2026). *PostgreSQL 18 documentation*.

PostgreSQL Global Development Group. (2026). *Date/time functions and operators*. https://www.postgresql.org/docs/18/functions-datetime.html

PostgreSQL Global Development Group. (2026). *PostgreSQL versioning policy*.
4 changes: 4 additions & 0 deletions docs/architecture/AS_BUILT_SCHEMA.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,10 @@ The protected-main slice persists:

The slice does **not** persist publication-event history, bound scientific evidence records, HTTP publication transport, or session-creation integration. Those remain Target unless separately evidenced on protected main.

## Active PR consent/outbox composition

Active PR #142 (`src/postgres_consent_propagation.rs`) adds no new physical objects. It composes the existing `consent_ledger` / `consent_event` slice with `integration_outbox` in one caller-owned `READ COMMITTED` transaction. After the submitted ledger snapshot is persisted, the adapter locks the participant `consent_ledger` row, requires every durable `consent_event` identity to appear in the submitted ledger, and requires a unique durable tail—ordered by `occurred_at_unix_ms`, then `created_at`—to match the outbox `causation_ref` and occurrence time. A grant-only in-memory snapshot therefore cannot enqueue grant propagation after a later stored revocation, and two rows that share occurrence time and `created_at` fail closed instead of using opaque `event_ref` as a correctness key. This slice is **Active PR**, not protected-main truth. Do not merge #70, #112, #120, #123, or #134.

## Logical-to-physical mapping rule

A logical entity is classified as physical only when all of the following exist on the named protected-main baseline:
Expand Down
2 changes: 1 addition & 1 deletion docs/architecture/ERD.md
Original file line number Diff line number Diff line change
Expand Up @@ -425,7 +425,7 @@ The target ERD deliberately includes several logical entities that are not yet p
- `instrument_release` is the locale-specific publication identity already owned by `src/instrument.rs`. Physical `migrations/0006_instrument_release.sql` persists that one-row aggregate (immutable manifest columns plus `publication_state`); HTTP publication transport remains Target.
- `data_rights_request` and `data_rights_propagation_state` are the first durable export/deletion slice. Physical `migrations/0003_data_rights_propagation.sql` stores requested-state identity plus one local outbox event per dependent system; verification, processing, completion, and dependent-system execution remain Target.
- `item_delivery_event` reflects the already-merged `src/item_delivery.rs` domain primitive; durable persistence/API orchestration is still Target.
- `consent_ledger` and `consent_event` persist the already-merged `src/consent.rs` append-only ledger. Physical persistence is carried by Active PR #49 (`migrations/0005_consent_lifecycle.sql`); HTTP consent transport and derived snapshot tables remain Target.
- `consent_ledger` and `consent_event` persist the already-merged `src/consent.rs` append-only ledger through `migrations/0005_consent_lifecycle.sql`. Active PR #142 composes those existing rows with `integration_outbox` in one caller-owned transaction after locking the participant ledger, requiring the submitted ledger to contain every durable event identity, and selecting a unique durable tail by occurrence time then `created_at`; it adds no new physical objects. HTTP consent transport and derived snapshot tables remain Target.
- `participant_identity_link` is the persistence target accepted by ADR-0020. The current `src/participant.rs` `keyverse_subject_ref` field is an application-domain first-link projection, not the future mutable persistence source of truth.
- `longitudinal_enrollment`, `longitudinal_observation_record`, and `temporal_analysis_submission` make the ADR-0008 Commons-owned Gyeot/TEPP orchestration boundary explicit. No TEPP analytical kernel is duplicated here.
- `integration_outbox`, `integration_delivery_attempt`, `integration_inbox`, and `integration_consumption` reflect `src/integration.rs` domain semantics. Outbox/inbox/delivery-attempt tables are on protected main; `integration_consumption` pending/processing/completed/quarantined persistence and expire-and-reclaim of a crashed processing claim exist only on this Active PR until merged.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This adjacent sentence still says integration_consumption exists only on this Active PR. Protected main already ships migrations/0012_integration_consumption.sql and TRACEABILITY maps that adapter as implemented. Leave it off this consent/outbox head; correct the leftover on a docs-only follow-up after #142 lands.

Expand Down
3 changes: 2 additions & 1 deletion docs/architecture/UML.md
Original file line number Diff line number Diff line change
Expand Up @@ -451,7 +451,8 @@ sequenceDiagram
participant S as semantic-data-portal

P->>A: explicit research opt-in for versioned scope
A->>DB: append consent evidence + research contribution
A->>DB: append consent evidence + bound outbox + research contribution
Note over A,DB: Active PR #142 commits the consent ledger change with its causally bound outbox row after locking the insertion-order durable ledger tail and failing closed on an ambiguous physical tail
DB-->>A: contribution_ref
A->>L: create/reuse scoped research pseudonym
L-->>A: research_participant_ref
Expand Down
1 change: 1 addition & 0 deletions src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ pub mod item_delivery;
pub mod narrative;
pub mod participant;
pub mod postgres_consent;
pub mod postgres_consent_propagation;
pub mod postgres_data_rights;
pub mod postgres_data_rights_processing;
pub mod postgres_health;
Expand Down
Loading
Loading