Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ All notable changes to TEPP are documented here. The format follows Keep a Chang

### Added

- `persistence_postgres` append-only immutability migration (`0004`): `reject_append_only_mutation`, statement-level `BEFORE UPDATE OR DELETE OR TRUNCATE` triggers on identity/manifest tables, `REVOKE UPDATE`/`DELETE`/`TRUNCATE` from `tepp_app_runtime`, executable DDL/rollback contracts, and live representative mutation proof.
- `persistence_postgres` model-run artifact chain: migration `0003_model_run_artifact_chain` for append-only `corpus_split_manifest`, `model_run`, and `model_artifact` with FORCE RLS; SQL insert/lookup contracts and live repository methods binding runs to reproducibility manifests and optional splits.
- `persistence_postgres` append-only reproducibility-manifest SQL contracts and live repository methods (`insert_reproducibility_manifest`, digest/id lookup) with fail-closed SHA-256 and commit identity validation for `reproducibility_manifest`.
- `persistence_postgres` tenant row-level security: migration `0002_tenant_row_level_security`, `tepp_app_runtime` role, session GUC `tepp.current_tenant_record_id`, multi-word isolation policies with FORCE RLS, session helpers, contract validation, and live isolation proof under `TEPP_LIVE_POSTGRES=1`.
Expand Down
7 changes: 7 additions & 0 deletions crates/persistence_postgres/src/error.rs
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,8 @@ pub enum MigrationContractError {
MissingAppRuntimeRole,
/// Tenant RLS was declared without the session tenant GUC contract.
MissingTenantSessionGuc,
/// Append-only immutability triggers or revoke statements were incomplete.
MissingAppendOnlyTrigger,
}

impl fmt::Display for MigrationContractError {
Expand All @@ -78,6 +80,7 @@ impl fmt::Display for MigrationContractError {
Self::MissingRlsPolicy => "missing tenant isolation policy",
Self::MissingAppRuntimeRole => "missing application runtime role",
Self::MissingTenantSessionGuc => "missing tenant session guc",
Self::MissingAppendOnlyTrigger => "missing append-only immutability trigger",
};
formatter.write_str(message)
}
Expand Down Expand Up @@ -159,5 +162,9 @@ mod tests {
MigrationContractError::MissingTenantSessionGuc.to_string(),
"missing tenant session guc"
);
assert_eq!(
MigrationContractError::MissingAppendOnlyTrigger.to_string(),
"missing append-only immutability trigger"
);
}
}
131 changes: 129 additions & 2 deletions crates/persistence_postgres/src/migration.rs
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,10 @@ const RLS_DOWN: &str = include_str!("../../../migrations/0002_tenant_row_level_s
const MODEL_RUN_UP: &str = include_str!("../../../migrations/0003_model_run_artifact_chain.up.sql");
const MODEL_RUN_DOWN: &str =
include_str!("../../../migrations/0003_model_run_artifact_chain.down.sql");
const APPEND_ONLY_UP: &str =
include_str!("../../../migrations/0004_append_only_immutability_triggers.up.sql");
const APPEND_ONLY_DOWN: &str =
include_str!("../../../migrations/0004_append_only_immutability_triggers.down.sql");

/// Forward and rollback SQL for one migration unit.
#[derive(Clone, Debug, Eq, PartialEq)]
Expand All @@ -28,8 +32,9 @@ impl MigrationCatalog {
/// Returns [`MigrationContractError::EmptyMigrationSql`] when embedded
/// sources are unexpectedly empty.
pub fn from_embedded() -> Result<Self, MigrationContractError> {
let up_sql = format!("{FOUNDATION_UP}\n{RLS_UP}\n{MODEL_RUN_UP}");
let down_sql = format!("{MODEL_RUN_DOWN}\n{RLS_DOWN}\n{FOUNDATION_DOWN}");
let up_sql = format!("{FOUNDATION_UP}\n{RLS_UP}\n{MODEL_RUN_UP}\n{APPEND_ONLY_UP}");
let down_sql =
format!("{APPEND_ONLY_DOWN}\n{MODEL_RUN_DOWN}\n{RLS_DOWN}\n{FOUNDATION_DOWN}");
Self::from_sources(&up_sql, &down_sql)
}

Expand Down Expand Up @@ -97,7 +102,40 @@ pub fn validate_migration_catalog(
if declares_row_level_security(catalog.up_sql()) {
validate_tenant_rls_contract(catalog.up_sql(), &tables)?;
}
if declares_append_only_immutability(catalog.up_sql()) {
validate_append_only_immutability(catalog.up_sql())?;
}

Ok(())
}

fn declares_append_only_immutability(up_sql: &str) -> bool {
let lower = up_sql.to_ascii_lowercase();
lower.contains("reject_append_only_mutation") || lower.contains("_reject_mutation")
}

fn validate_append_only_immutability(up_sql: &str) -> Result<(), MigrationContractError> {
let lower = up_sql.to_ascii_lowercase();
if !lower.contains("create or replace function reject_append_only_mutation") {
return Err(MigrationContractError::MissingAppendOnlyTrigger);
}
let required = [
"source_artifact",
"audit_event",
"reproducibility_manifest",
"corpus_split_manifest",
"model_run",
"model_artifact",
];
for table in required {
let trigger = format!("{table}_reject_mutation");
if !lower.contains(&format!("create trigger {trigger}")) {
return Err(MigrationContractError::MissingAppendOnlyTrigger);
}
if !lower.contains(&format!("revoke update, delete on table {table}")) {
return Err(MigrationContractError::MissingAppendOnlyTrigger);
}
}
Ok(())
}

Expand Down Expand Up @@ -548,6 +586,95 @@ mod tests {
);
}

#[test]
fn append_only_immutability_contract_fails_closed() {
let missing_function = MigrationCatalog::from_sql(
r"
CREATE TABLE tenant_record (
tenant_record_id uuid PRIMARY KEY,
system_time timestamptz NOT NULL
);
CREATE TRIGGER source_artifact_reject_mutation
BEFORE UPDATE ON source_artifact
FOR EACH ROW EXECUTE FUNCTION reject_append_only_mutation();
",
"DROP TABLE tenant_record;",
);
assert_eq!(
validate_migration_catalog(&missing_function),
Err(MigrationContractError::MissingAppendOnlyTrigger)
);

let missing_trigger = MigrationCatalog::from_sql(
r"
CREATE TABLE tenant_record (
tenant_record_id uuid PRIMARY KEY,
system_time timestamptz NOT NULL
);
CREATE OR REPLACE FUNCTION reject_append_only_mutation()
RETURNS trigger LANGUAGE plpgsql AS $$ BEGIN RETURN NEW; END $$;
",
"DROP TABLE tenant_record;",
);
assert_eq!(
validate_migration_catalog(&missing_trigger),
Err(MigrationContractError::MissingAppendOnlyTrigger)
);

// All triggers present; REVOKE omitted only for model_artifact so the
// last revoke branch returns MissingAppendOnlyTrigger.
let missing_revoke = MigrationCatalog::from_sql(
r"
CREATE TABLE tenant_record (
tenant_record_id uuid PRIMARY KEY,
system_time timestamptz NOT NULL
);
CREATE OR REPLACE FUNCTION reject_append_only_mutation()
RETURNS trigger LANGUAGE plpgsql AS $$ BEGIN RETURN NEW; END $$;
CREATE TRIGGER source_artifact_reject_mutation
BEFORE UPDATE ON source_artifact
FOR EACH ROW EXECUTE FUNCTION reject_append_only_mutation();
REVOKE UPDATE, DELETE ON TABLE source_artifact FROM tepp_app_runtime;
CREATE TRIGGER audit_event_reject_mutation
BEFORE UPDATE ON audit_event
FOR EACH ROW EXECUTE FUNCTION reject_append_only_mutation();
REVOKE UPDATE, DELETE ON TABLE audit_event FROM tepp_app_runtime;
CREATE TRIGGER reproducibility_manifest_reject_mutation
BEFORE UPDATE ON reproducibility_manifest
FOR EACH ROW EXECUTE FUNCTION reject_append_only_mutation();
REVOKE UPDATE, DELETE ON TABLE reproducibility_manifest FROM tepp_app_runtime;
CREATE TRIGGER corpus_split_manifest_reject_mutation
BEFORE UPDATE ON corpus_split_manifest
FOR EACH ROW EXECUTE FUNCTION reject_append_only_mutation();
REVOKE UPDATE, DELETE ON TABLE corpus_split_manifest FROM tepp_app_runtime;
CREATE TRIGGER model_run_reject_mutation
BEFORE UPDATE ON model_run
FOR EACH ROW EXECUTE FUNCTION reject_append_only_mutation();
REVOKE UPDATE, DELETE ON TABLE model_run FROM tepp_app_runtime;
CREATE TRIGGER model_artifact_reject_mutation
BEFORE UPDATE ON model_artifact
FOR EACH ROW EXECUTE FUNCTION reject_append_only_mutation();
",
"DROP TABLE tenant_record;",
);
assert_eq!(
validate_migration_catalog(&missing_revoke),
Err(MigrationContractError::MissingAppendOnlyTrigger)
);

assert!(super::declares_append_only_immutability(
"CREATE TRIGGER source_artifact_reject_mutation"
));
assert!(!super::declares_append_only_immutability("CREATE TABLE x"));
assert_eq!(
super::validate_append_only_immutability(
"CREATE TRIGGER source_artifact_reject_mutation BEFORE UPDATE ON source_artifact \
FOR EACH ROW EXECUTE FUNCTION reject_append_only_mutation();"
),
Err(MigrationContractError::MissingAppendOnlyTrigger)
);
}

#[test]
fn empty_and_malformed_sql_fail_closed() {
let empty = MigrationCatalog::from_sql(" ", "DROP TABLE x;");
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
//! Append-only migration DDL must block every destructive table operation.

use persistence_postgres::{MigrationCatalog, validate_migration_catalog};

const APPEND_ONLY_TABLES: [&str; 6] = [
"source_artifact",
"audit_event",
"reproducibility_manifest",
"corpus_split_manifest",
"model_run",
"model_artifact",
];

fn normalized(sql: &str) -> String {
sql.split_whitespace()
.collect::<Vec<_>>()
.join(" ")
.to_ascii_lowercase()
}

#[test]
fn embedded_append_only_ddl_rejects_update_delete_and_truncate() {
let catalog = MigrationCatalog::from_embedded().expect("embedded migration catalog");
validate_migration_catalog(&catalog).expect("embedded migration contract");

let up_sql = normalized(catalog.up_sql());
assert!(up_sql.contains("raise exception 'append-only table % rejects %'"));
assert!(up_sql.contains("errcode = 'integrity_constraint_violation'"));

for table in APPEND_ONLY_TABLES {
assert!(
up_sql.contains(&format!(
"revoke update, delete on table {table} from tepp_app_runtime"
)),
"runtime role must not retain UPDATE/DELETE on {table}"
);
assert!(
up_sql.contains(&format!(
"revoke truncate on table {table} from tepp_app_runtime"
)),
"runtime role must not retain TRUNCATE on {table}"
);
assert!(
up_sql.contains(&format!(
"create trigger {table}_reject_mutation before update or delete or truncate on {table} for each statement execute function reject_append_only_mutation()"
)),
"statement trigger must reject UPDATE, DELETE, and TRUNCATE on {table}"
);
}
}

#[test]
fn rollback_removes_rejection_triggers_without_granting_truncate() {
let catalog = MigrationCatalog::from_embedded().expect("embedded migration catalog");
let down_sql = normalized(catalog.down_sql());

assert!(down_sql.contains("drop function if exists reject_append_only_mutation()"));
assert!(down_sql.contains(
"drop trigger if exists %i on %i', trigger_table || '_reject_mutation', trigger_table"
));
for table in APPEND_ONLY_TABLES {
assert!(
down_sql.contains(&format!("'{table}'")),
"rollback trigger inventory must include {table}"
);
assert!(
down_sql.contains(&format!(
"grant update, delete on table {table} to tepp_app_runtime"
)),
"rollback must restore only the privileges granted before migration 0004 on {table}"
);
assert!(
!down_sql.contains(&format!(
"grant truncate on table {table} to tepp_app_runtime"
)),
"rollback must not introduce a new TRUNCATE privilege on {table}"
);
}
}
26 changes: 26 additions & 0 deletions crates/persistence_postgres/tests/live_postgres.rs
Original file line number Diff line number Diff line change
Expand Up @@ -176,9 +176,35 @@ fn live_postgres_applies_migrations_and_document_sql() {
.expect("select reproducibility_manifest by id");

exercise_model_run_artifact_chain(&mut repo, tenant_record_id, &manifest, available);
prove_append_only_immutability(&mut repo, &manifest);
prove_tenant_rls_isolation(&mut repo);
}

/// Append-only triggers must reject UPDATE/DELETE on identity tables.
fn prove_append_only_immutability(
repo: &mut LiveDocumentRepository<persistence_postgres::LiveSqlxPool>,
manifest: &ReproducibilityManifestRecord,
) {
let update = format!(
"UPDATE reproducibility_manifest SET code_commit_sha = 'deadbeef' \
WHERE reproducibility_manifest_id = '{}'::uuid",
manifest.reproducibility_manifest_id
);
assert!(
repo.session_mut().execute(&update).is_err(),
"UPDATE must fail on append-only reproducibility_manifest"
);
let delete = format!(
"DELETE FROM reproducibility_manifest \
WHERE reproducibility_manifest_id = '{}'::uuid",
manifest.reproducibility_manifest_id
);
assert!(
repo.session_mut().execute(&delete).is_err(),
"DELETE must fail on append-only reproducibility_manifest"
);
}

fn exercise_model_run_artifact_chain(
repo: &mut LiveDocumentRepository<persistence_postgres::LiveSqlxPool>,
tenant_record_id: Uuid,
Expand Down
6 changes: 3 additions & 3 deletions docs/ERD.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
**Status:** Accepted logical target model with current implementation maturity explicitly marked.
**Last reviewed:** 2026-08-13

Protected main implements storage-independent domain objects plus `persistence_postgres` foundation tables (`0001`) and tenant row-level security (`0002`) as executable migration contracts with live CI. Broader planned ERD entities, exactly-one membership constraints, and backup/recovery gates remain accepted-target until each lands with exact-head evidence.
Protected main implements storage-independent domain objects plus `persistence_postgres` foundation tables (`0001`), tenant row-level security (`0002`), and the model-run/artifact chain (`0003`) as executable migration contracts with live CI. Migration `0004` adds append-only mutation controls on the active PR and must not be treated as implemented-main until exact-head checks, review, and protected-main integration complete. Broader planned ERD entities, exactly-one membership constraints, concurrent-write acceptance, and backup/recovery gates remain accepted-target until each lands with exact-head evidence.

## Current domain foundation

Expand Down Expand Up @@ -355,7 +355,7 @@ Every `MODEL_RUN` binds two immutable identities:
- `corpus_split_manifest_id`: the exact relation-aware train/validation/test split, including the relation-component digest, split policy version, partition hashes, and knowledge cutoff used to prevent translation/revision/episode leakage;
- `reproducibility_manifest_id`: the exact source/evidence manifests, preprocessing and concept-dictionary versions, model contract/configuration, dependency lock, Git commit, and provenance-manifest identity used for the run.

Both manifest tables are append-only identity records. Their `canonical_payload_hash` is the lowercase SHA-256 digest of a versioned, deterministically encoded payload containing every identity-bearing field; `split_manifest_hash` and `reproducibility_manifest_hash` remain the public domain-specific identities and must match that canonical payload under their declared algorithm version. Database roles deny UPDATE and DELETE, and a defense-in-depth trigger rejects either operation. When `protected_object_ref` is present, it addresses a versioned immutable object; every read recomputes and compares the object digest before trusting its payload. A missing object, mutable reference, digest mismatch, or changed payload fails closed.
Both manifest tables are append-only identity records. Their `canonical_payload_hash` is the lowercase SHA-256 digest of a versioned, deterministically encoded payload containing every identity-bearing field; `split_manifest_hash` and `reproducibility_manifest_hash` remain the public domain-specific identities and must match that canonical payload under their declared algorithm version. Migration `0004` applies defense in depth by revoking `UPDATE`, `DELETE`, and `TRUNCATE` from the application runtime role and installing statement-level `BEFORE UPDATE OR DELETE OR TRUNCATE` triggers on governed identity/manifest tables. These controls prevent ordinary application-role mutation and trip owner-session mistakes inside the governed migration lifecycle; they do not claim to resist a superuser or owner who deliberately drops or disables the controls. When `protected_object_ref` is present, it addresses a versioned immutable object; every read recomputes and compares the object digest before trusting its payload. A missing object, mutable reference, digest mismatch, or changed payload fails closed.

`MODEL_RUN.random_seed_manifest_hash` resolves to an immutable, digest-verified seed manifest that fixes every model/sampler seed without exposing secret entropy in ordinary logs. `MODEL_RUN.compute_backend_code` fixes the governed CPU/GPU backend contract used by that run, including the referenced implementation/version evidence. These fields are part of the run identity and must agree with the referenced reproducibility manifest's configuration payload.

Expand All @@ -369,4 +369,4 @@ Every published analytical artifact must be traceable to source hashes, evidence

## Migration acceptance

Before this planned ERD becomes as-built, migrations must include rollback, tenant/RLS policy, temporal relation constraints/indexes, exactly-one membership constraints, relation-aware split/manifests, lineage integrity, idempotency/concurrency, retention/deletion, backup/recovery, and synthetic known-truth integration tests. The documentation maturity label then changes only after protected-main integration and exact-current-head evidence.
Before the remaining planned ERD becomes as-built, migrations must include rollback, tenant/RLS policy, temporal relation constraints/indexes, exactly-one membership constraints, relation-aware split/manifests, lineage integrity, idempotency/concurrency, retention/deletion, backup/recovery, and synthetic known-truth integration tests. The documentation maturity label then changes only after protected-main integration and exact-current-head evidence.
Loading
Loading