diff --git a/CONTEXT.md b/CONTEXT.md index 24d4fd26..7eb14cb2 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -329,7 +329,9 @@ registered service workload，并防 cross-job/cross-tenant replay。 - **Owner/scope:** one Organization, one durable job attempt, its declared work, and the registered service workload that may perform it. - **Lifecycle:** short-lived and one-shot; mismatch, expiry, staleness, or replay - makes it invalid. Exact claim/redemption fields belong to the owning ADR. + makes it invalid. A reclaimed job receives a higher signed lease generation, + so the replaced generation and nonce cannot resume or mutate it. Exact + claim/redemption fields belong to the owning ADR. - **Invariant:** no general tenant/read/action authority and no long-lived source credential; rejected lease produces zero business effect. - **Activation note:** Issue #17 binds a registered ServicePrincipal to @@ -372,6 +374,36 @@ acquisition 精确绑定。 - **Do not confuse with:** active ContextRevision, publish watermark, acquisition checkpoint, or recovery lease. +### `File publication recovery checkpoint` + +The mutable durable step marker for one interrupted File publication. 中文：File +publication recovery checkpoint 将同一个 job 固定到唯一 Resource、Revision、内容 +identity，并允许新租约从已提交边界继续。 + +- **Owner/scope:** one Organization, File import job, ContextSource, + ContextResource, and stable target ContextRevision. +- **Lifecycle:** advances `acquired -> prepared -> ready -> completed`; an + expired attempt may be reclaimed only by a higher signed lease generation. +- **Invariant:** it neither authorizes delivery nor makes a Revision active; + every resumed step and activation revalidates the current lease generation, + exact compilation/write-artifact digest, and current authority. +- **Do not confuse with:** File acquisition outcome, replacement plan, + publication event, active pointer, or automatic retry scheduler. + +### `File import job event` + +The immutable ordered audit of File publication boundaries and recovery. 中文： +File import job event 记录中断、租约回收及最终激活，且不保存源内容。 + +- **Owner/scope:** one Organization and File import job, with lease generation + and optional stable Revision lineage. +- **Lifecycle:** append-only events for acquired, prepared, indexed, + interrupted, reclaimed, unchanged, and active transitions. +- **Invariant:** interruption/reclaim reasons are fixed-category digests; the + event is audit evidence, never execution or content authority. +- **Do not confuse with:** Revision publication event, WorkerLease, queue + delivery, checkpoint, or retry policy. + ### `File revision supersession` The immutable old-to-new Revision edge recorded by successful File activation. diff --git a/docs/decisions/0041-recover-file-publication-by-durable-boundary.md b/docs/decisions/0041-recover-file-publication-by-durable-boundary.md new file mode 100644 index 00000000..b67633f0 --- /dev/null +++ b/docs/decisions/0041-recover-file-publication-by-durable-boundary.md @@ -0,0 +1,85 @@ +--- +name: adr-0041-recover-file-publication-by-durable-boundary +version: "1.0.0" +description: > + Recover one interrupted File publication from stable acquired, prepared, or + ready state using a higher-generation lease and immutable transition audit. +--- + +# 0041. Recover File publication by durable boundary + +- Status: accepted +- Date: 2026-07-23 +- Refines: ADR-0029, ADR-0037, ADR-0039, ADR-0040 + +## Context + +File publication now spans classification, immutable Revision construction, +candidate/index preparation, and the active-pointer transaction. A worker may +stop after any committed step. Deleting partial state and creating a new job +would lose the exact attempt identity, can duplicate immutable lineage, and can +create an empty visibility window during replacement. + +Recovery must reuse the File job and WorkerLease protocol. It must also make an +expired lease permanently stale once a new worker owns the job, while keeping +the old active Revision available until activation. + +## Decision + +One File job has four durable checkpoints: `acquired`, `prepared`, `ready`, and +`completed`. `file_publication_recovery` binds the job to Organization, Source, +stable Resource, one generated Revision, optional previous Revision, versioned +content identity, compiler contract, and a digest of the exact compilation and +write artifacts. Structural-v2 artifacts must exactly project the validated +compilation document; v1 artifacts must exactly project the canonical paragraph. +Each step is a separate PostgreSQL transaction and advances only from its exact +predecessor. + +Lease issue may reclaim an expired `leased`, `running`, `prepared`, or `ready` +job. It increments `lease_generation`, replaces the nonce/time binding, and +records the state to resume. The generation is a signed WorkerLease claim; +redemption restores that state and clears the transient resume marker. Every +later mutation, including compatibility publication seams, rechecks the current +exact generation and lease; an old nonce or generation can perform no work. + +`file_import_job_event` is immutable and records acquired, prepared, indexed, +explicit interruption, reclaim, unchanged, and active transitions. Failure and +reclaim evidence uses fixed categories plus digests; it stores no source +content. Deterministic test interruption is injected only after a committed +boundary and is never auto-retried. + +The Organization/Source/Resource ingestion guard serializes initial and changed +classification. A concurrent equivalent winner is re-observed as an auditable +zero-effect no-op. A second worker cannot redeem the same lease. Recovery +reuses the stable Revision and existing rows rather than inserting them again. + +Activation revalidates the current ServicePrincipal, acquisition Membership, +body access, complete Fragment/candidate representation, and ordered +publication evidence. Initial activation changes a null active pointer once; +replacement delegates to ADR-0040's previous-to-new compare-and-swap. Therefore +Runtime sees the complete old Revision until the complete recovered Revision is +activated. + +## Consequences + +- The three named interruption boundaries resume to one active Revision without + duplicate Resource, Revision, Fragment, candidate, job, or Package evidence. +- A durable checkpoint is retained for later reclaim; pre-checkpoint authority + or compilation failures remain terminal and content-free. +- Recovery rows and job history are tenant-owned, FORCE-RLS protected, and + accessible to the worker only through SECURITY DEFINER functions. +- A recovery schema with any non-completed checkpoint or job event attached to + a non-completed job is intentionally non-downgradable because removing it + would erase the only resume contract or audit. Completed checkpoint/audit + metadata may be removed by an explicit schema downgrade because the immutable + publication lineage remains in the Issue #26 tables. +- Arbitrary instruction-level chaos, batch recovery, delete/tombstone recovery, + dead-letter handling, and operator remediation remain inactive. + +## Revisit trigger + +Revisit before automatic retry scheduling, explicit lease release, batch or +delete recovery, checkpoint compaction, dead-letter processing, or physical +cleanup. Any change must preserve exact Organization/job/Revision identity, +higher-generation lease fencing, audit continuity, and all-old/all-new Runtime +visibility. diff --git a/docs/decisions/README.md b/docs/decisions/README.md index 67935411..72cd3d1a 100644 --- a/docs/decisions/README.md +++ b/docs/decisions/README.md @@ -127,3 +127,4 @@ touched: - [0038 — Structural Markdown units](0038-compile-and-publish-structural-markdown.md) - [0039 — File acquisition no-op](0039-deduplicate-unchanged-file-acquisitions.md) - [0040 — Atomic File replacement](0040-stage-and-atomically-activate-file-replacements.md) +- [0041 — Durable File publication recovery](0041-recover-file-publication-by-durable-boundary.md) diff --git a/docs/design/2026-07-18-context-engine-implementation-design.md b/docs/design/2026-07-18-context-engine-implementation-design.md index f4058100..4a161392 100644 --- a/docs/design/2026-07-18-context-engine-implementation-design.md +++ b/docs/design/2026-07-18-context-engine-implementation-design.md @@ -561,14 +561,27 @@ durable replacement plan before marking the job `ready`. The old Revision stays active throughout staging. A second transaction revalidates authority and readiness, compare-and-swaps the Resource active pointer, appends `active`, records immutable supersession lineage, and completes the job. Superseded -artifacts remain `retained_until_explicit_cleanup`; deletion and recovery of a -committed ready job are not active in this slice. When an equivalent concurrent +artifacts remain `retained_until_explicit_cleanup`; deletion is not active in +this slice. When an equivalent concurrent replacement activates between the initial publish attempt and replacement staging, the guarded stage classification completes the later job as `unchanged` and returns that durable zero-effect result. V1 and V2 each reprove that the supplied compilation exactly matches the now-active snapshot, Fragments, and candidates before reporting success. +ADR-0041 makes publication recoverable after exactly the committed `acquired`, +`prepared`, and `ready` boundaries. One tenant-owned checkpoint binds the +existing File job to its stable Resource/Revision/content identity, while an +immutable job-event stream records interruption, reclaim, and completion. +Reclaim issues a higher signed lease generation only after expiry; the replaced +generation and nonce cannot resume or mutate, including through compatibility +publication seams. Each idempotent step revalidates the exact current lease and +advances one checkpoint. Initial and replacement activation recheck +current audience authority and complete Fragment/candidate evidence before the +single pointer transaction, so a recovered replacement leaves old active +content visible until the new Revision is complete. Automatic retry scheduling, +arbitrary chaos, delete recovery, and dead-letter handling remain inactive. + Because Runtime resolves through multiple SQL statements at `READ COMMITTED`, each UserActor transaction takes an Organization-scoped shared publication barrier and activation takes the matching exclusive transaction barrier around @@ -832,8 +845,7 @@ versioned public wire and activates breaking-change checks. Exit: - Markdown AST, hash incrementality, delete detection, checkpoint replay, - publication crash recovery, lease reclaim, dead-letter, and full-resync - runbook are complete; + broader crash recovery, dead-letter, and full-resync runbook are complete; - before experiment execution, the frozen dataset registers a sample plan based on failure-slice coverage and a declared uncertainty/power target; negative cases cover every active refusal/security category, and an underpowered set is diff --git a/engine/persistence/__init__.py b/engine/persistence/__init__.py index 5bc5871a..8d74620d 100644 --- a/engine/persistence/__init__.py +++ b/engine/persistence/__init__.py @@ -34,8 +34,10 @@ from engine.persistence.control_sources import PostgreSQLControlStore from engine.persistence.database import create_database_engine from engine.persistence.file_imports import ( + FileImportInterrupted, FileImportLeaseRedemption, FileImportUnavailable, + FilePublicationBoundary, PostgreSQLFileImportWorker, PublishedFileImport, ) @@ -99,7 +101,9 @@ "PostgreSQLContextRunReader", "PostgreSQLControlStore", "FileImportLeaseRedemption", + "FileImportInterrupted", "FileImportUnavailable", + "FilePublicationBoundary", "PostgreSQLFileImportWorker", "PublishedFileImport", "VerifiedContextRunOperatorIdentity", diff --git a/engine/persistence/file_imports.py b/engine/persistence/file_imports.py index 4caf3e5e..01c250cc 100644 --- a/engine/persistence/file_imports.py +++ b/engine/persistence/file_imports.py @@ -3,8 +3,11 @@ from __future__ import annotations import json +import time from collections.abc import Callable +from contextlib import suppress from dataclasses import dataclass, field +from enum import StrEnum from hashlib import sha256 from typing import Literal from uuid import UUID, uuid4 @@ -21,7 +24,6 @@ SourceRef, ) from engine.persistence.role_guard import assert_worker_role -from engine.runtime.content_io import exact_phrase_digest from engine.runtime.evidence import CandidateRef from engine.supply import ( FILE_IMPORT_WORKER_LEASE_OPERATION, @@ -38,6 +40,9 @@ ) from engine.supply.jobs import _require_utc +_CONCURRENT_PUBLICATION_WAIT_SECONDS = 5.0 +_CONCURRENT_PUBLICATION_POLL_SECONDS = 0.01 + @dataclass(frozen=True, slots=True) class FileImportLeaseRedemption: @@ -139,6 +144,24 @@ class FileImportUnavailable(RuntimeError): """Generic failure after a valid lease reaches acquisition/publication.""" +class FilePublicationBoundary(StrEnum): + """The three explicit post-commit fault-injection boundaries.""" + + ACQUIRED = "acquired" + PREPARED = "prepared" + INDEXED = "indexed" + + +class FileImportInterrupted(RuntimeError): + """Deterministic test interruption recorded after a durable boundary.""" + + def __init__(self, boundary: FilePublicationBoundary) -> None: + if type(boundary) is not FilePublicationBoundary: + raise TypeError("File interruption requires an exact boundary") + self.boundary = boundary + super().__init__(f"File publication interrupted after {boundary.value}") + + @dataclass(frozen=True, slots=True) class _RedeemedFileImport: source_ref: SourceRef @@ -171,6 +194,7 @@ class PostgreSQLFileImportWorker: "_config", "_engine", "_identity", + "_interrupt_after", "_roots", "_uuid_factory", ) @@ -185,6 +209,7 @@ def __init__( *, clock: Callable[[], object], uuid_factory: Callable[[], UUID] = uuid4, + interrupt_after: FilePublicationBoundary | None = None, ) -> None: if type(codec) is not WorkerLeaseCodec: raise TypeError("File import worker requires WorkerLeaseCodec") @@ -196,6 +221,11 @@ def __init__( raise TypeError("File import worker requires MarkdownCompilerConfig") if not callable(clock) or not callable(uuid_factory): raise TypeError("File import worker requires clock and UUID factory") + if ( + interrupt_after is not None + and type(interrupt_after) is not FilePublicationBoundary + ): + raise TypeError("File import interruption requires a closed boundary") self._engine = engine self._codec = codec self._identity = identity @@ -203,6 +233,7 @@ def __init__( self._config = config self._clock = clock self._uuid_factory = uuid_factory + self._interrupt_after = interrupt_after def run(self, redemption: FileImportLeaseRedemption) -> PublishedFileImport: """Perform file I/O only after signature and durable lease redemption.""" @@ -237,8 +268,11 @@ def run(self, redemption: FileImportLeaseRedemption) -> PublishedFileImport: raise FileImportUnavailable("File import is unavailable") try: return self._publish(redemption.token, claims, redeemed, outcome) + except FileImportInterrupted: + raise except (FileImportUnavailable, WorkNotAvailable): - self._fail(redemption.token, claims) + with suppress(FileImportUnavailable, WorkNotAvailable): + self._fail(redemption.token, claims) raise def _redeem( @@ -256,7 +290,8 @@ def _redeem( """ SELECT * FROM public.context_worker_redeem_file_import( :organization_id, :job_id, :service_principal_id, - :source_ref, :signing_key_version, :nonce, + :source_ref, :lease_generation, + :signing_key_version, :nonce, :issued_at, :expires_at ) """ @@ -266,6 +301,7 @@ def _redeem( "job_id": claims.job_id, "service_principal_id": claims.service_principal_id, "source_ref": claims.source_ref, + "lease_generation": claims.lease_generation, "signing_key_version": claims.signing_key_version, "nonce": claims.nonce, "issued_at": claims.issued_at, @@ -296,83 +332,137 @@ def _publish( ) -> PublishedFileImport: if type(claims) is not WorkerLeaseClaims: raise _rejection(token) - revision_id = self._uuid_factory() + requested_revision_id = self._uuid_factory() resource_ref = _resource_ref(redeemed.source_ref, redeemed.path) - structural = document.provenance.is_structural_v2 - if structural: - compilation_document = json.loads( + if document.provenance.is_structural_v2: + raw_compilation_document = json.loads( canonicalize_parsed_document(document).decode("utf-8") ) - statement = """ - SELECT * - FROM public.context_worker_publish_structural_file_import_v2( + compilation_document: str | None = json.dumps( + raw_compilation_document, + ensure_ascii=False, + separators=(",", ":"), + ) + else: + compilation_document = None + artifact_document = json.dumps( + [ + { + "fragmentRef": fragment.fragment_ref, + "contextualText": fragment.contextual_text, + "searchPhrases": list(fragment.search_phrases), + } + for fragment in document.fragments + ], + ensure_ascii=False, + separators=(",", ":"), + ) + parameters: dict[str, object] = { + "organization_id": claims.organization_id, + "job_id": claims.job_id, + "service_principal_id": claims.service_principal_id, + "source_ref": claims.source_ref, + "resource_ref": resource_ref, + "revision_id": requested_revision_id, + "canonical_text": document.canonical_text, + "content_hash": document.content_hash, + "compilation_digest": document.compilation_digest, + "compiler_version": document.provenance.compiler_version, + "config_version": document.provenance.config_version, + "compilation_document": compilation_document, + "artifact_document": artifact_document, + "lease_generation": claims.lease_generation, + "signing_key_version": claims.signing_key_version, + "nonce": claims.nonce, + "issued_at": claims.issued_at, + "expires_at": claims.expires_at, + } + try: + acquired = self._execute_one( + """ + SELECT * FROM public.context_worker_acquire_file_publication( :organization_id, :job_id, :service_principal_id, - :source_ref, :resource_ref, - :revision_id, :canonical_text, - :content_hash, :compilation_digest, + :source_ref, :resource_ref, :revision_id, + :canonical_text, :content_hash, :compilation_digest, :compiler_version, :config_version, CAST(:compilation_document AS jsonb), - :signing_key_version, :nonce, :issued_at, :expires_at + CAST(:artifact_document AS jsonb), + :lease_generation, :signing_key_version, :nonce, + :issued_at, :expires_at ) - """ - payload: dict[str, object] = { - "compilation_document": json.dumps( - compilation_document, - ensure_ascii=False, - separators=(",", ":"), - ) - } - else: - fragment = document.fragments[0] - statement = """ - SELECT * - FROM public.context_worker_publish_file_import_v2( - :organization_id, :job_id, :service_principal_id, - :source_ref, :resource_ref, - :revision_id, :fragment_ref, :canonical_text, - :paragraph, :content_hash, :compilation_digest, - :compiler_version, :config_version, :phrase_digest, - :signing_key_version, :nonce, :issued_at, :expires_at + """, + parameters, + ) + if acquired is None: + raise _rejection(token) + if acquired.checkpoint == "contended": + acquired = self._await_concurrent_publication(token, parameters) + if acquired.outcome == "unchanged": + row = acquired + else: + parameters["revision_id"] = acquired.stable_revision_id + if acquired.checkpoint == "acquired": + self._interrupt_if_requested( + token, claims, FilePublicationBoundary.ACQUIRED + ) + prepared = self._execute_one( + """ + SELECT * FROM public.context_worker_prepare_file_publication( + :organization_id, :job_id, :service_principal_id, + :source_ref, :resource_ref, :revision_id, + :canonical_text, + CAST(:compilation_document AS jsonb), + CAST(:artifact_document AS jsonb), + :lease_generation, :signing_key_version, :nonce, + :issued_at, :expires_at + ) + """, + parameters, + ) + if prepared is None or prepared.checkpoint != "prepared": + raise _rejection(token) + acquired = prepared + if acquired.checkpoint == "prepared": + self._interrupt_if_requested( + token, claims, FilePublicationBoundary.PREPARED + ) + indexed = self._execute_one( + """ + SELECT * FROM public.context_worker_index_file_publication( + :organization_id, :job_id, :service_principal_id, + :source_ref, :resource_ref, :revision_id, + :canonical_text, + CAST(:compilation_document AS jsonb), + CAST(:artifact_document AS jsonb), + :lease_generation, :signing_key_version, :nonce, + :issued_at, :expires_at + ) + """, + parameters, + ) + if indexed is None or indexed.checkpoint != "ready": + raise _rejection(token) + acquired = indexed + if acquired.checkpoint != "ready": + raise _rejection(token) + self._interrupt_if_requested( + token, claims, FilePublicationBoundary.INDEXED ) - """ - payload = { - "fragment_ref": fragment.fragment_ref, - "paragraph": fragment.contextual_text, - "phrase_digest": exact_phrase_digest(fragment.search_phrases[0]), - } - try: - with self._engine.begin() as connection: - assert_worker_role(connection) - row = connection.execute( - text(statement), - { - "organization_id": claims.organization_id, - "job_id": claims.job_id, - "service_principal_id": claims.service_principal_id, - "source_ref": claims.source_ref, - "resource_ref": resource_ref, - "revision_id": revision_id, - "canonical_text": document.canonical_text, - "content_hash": document.content_hash, - "compilation_digest": document.compilation_digest, - "compiler_version": document.provenance.compiler_version, - "config_version": document.provenance.config_version, - "signing_key_version": claims.signing_key_version, - "nonce": claims.nonce, - "issued_at": claims.issued_at, - "expires_at": claims.expires_at, - **payload, - }, - ).one_or_none() - if row is None: - row = self._replace( - claims, - resource_ref, - revision_id, - document, - payload, - structural=structural, + activated = self._execute_one( + """ + SELECT * + FROM public.context_worker_activate_recoverable_file_publication( + :organization_id, :job_id, :service_principal_id, + :source_ref, :resource_ref, :revision_id, + :lease_generation, :signing_key_version, :nonce, + :issued_at, :expires_at + ) + """, + parameters, ) + if activated is None: + raise _rejection(token) + row = activated if ( row is None or row.effect_count not in {0, 1} @@ -381,7 +471,7 @@ def _publish( or not row.fragment_refs ): raise _rejection(token) - except WorkNotAvailable: + except (WorkNotAvailable, FileImportInterrupted): raise except (SQLAlchemyError, AssertionError): raise FileImportUnavailable("File publication is unavailable") from None @@ -403,86 +493,62 @@ def _publish( effect_count=row.effect_count, ) - def _replace( + def _await_concurrent_publication( self, - claims: WorkerLeaseClaims, - resource_ref: str, - revision_id: UUID, - document: ParsedDocument, - payload: dict[str, object], - *, - structural: bool, - ) -> Row[tuple[object, ...]] | None: - if structural: - stage_statement = """ - SELECT * - FROM public.context_worker_stage_structural_file_replacement( + token: WorkerLeaseToken, + parameters: dict[str, object], + ) -> Row[tuple[object, ...]]: + """Let a concurrently committed winner become this job's no-op input.""" + + deadline = time.monotonic() + _CONCURRENT_PUBLICATION_WAIT_SECONDS + while time.monotonic() < deadline: + time.sleep(_CONCURRENT_PUBLICATION_POLL_SECONDS) + acquired = self._execute_one( + """ + SELECT * FROM public.context_worker_acquire_file_publication( :organization_id, :job_id, :service_principal_id, :source_ref, :resource_ref, :revision_id, :canonical_text, :content_hash, :compilation_digest, :compiler_version, :config_version, CAST(:compilation_document AS jsonb), - :signing_key_version, :nonce, :issued_at, :expires_at + CAST(:artifact_document AS jsonb), + :lease_generation, :signing_key_version, :nonce, + :issued_at, :expires_at ) - """ - else: - stage_statement = """ - SELECT * - FROM public.context_worker_stage_file_replacement( - :organization_id, :job_id, :service_principal_id, - :source_ref, :resource_ref, :revision_id, - :fragment_ref, :canonical_text, :paragraph, - :content_hash, :compilation_digest, - :compiler_version, :config_version, :phrase_digest, - :signing_key_version, :nonce, :issued_at, :expires_at - ) - """ + """, + parameters, + ) + if acquired is None: + raise _rejection(token) + if acquired.checkpoint != "contended": + return acquired + raise _rejection(token) + + def _execute_one( + self, + statement: str, + parameters: dict[str, object], + ) -> Row[tuple[object, ...]] | None: with self._engine.begin() as connection: assert_worker_role(connection) - staged = connection.execute( - text(stage_statement), - { - "organization_id": claims.organization_id, - "job_id": claims.job_id, - "service_principal_id": claims.service_principal_id, - "source_ref": claims.source_ref, - "resource_ref": resource_ref, - "revision_id": revision_id, - "canonical_text": document.canonical_text, - "content_hash": document.content_hash, - "compilation_digest": document.compilation_digest, - "compiler_version": document.provenance.compiler_version, - "config_version": document.provenance.config_version, - "signing_key_version": claims.signing_key_version, - "nonce": claims.nonce, - "issued_at": claims.issued_at, - "expires_at": claims.expires_at, - **payload, - }, - ).one_or_none() - if staged is None: - return None - if staged.outcome == "unchanged": - return staged - if ( - staged.outcome is not None - or staged.effect_count is not None - or staged.active_revision_id is not None - or staged.reason_digest is not None - or staged.previous_revision_id is None - or staged.replacement_revision_id is None - ): - return None + return connection.execute(text(statement), parameters).one_or_none() + + def _interrupt_if_requested( + self, + token: WorkerLeaseToken, + claims: WorkerLeaseClaims, + boundary: FilePublicationBoundary, + ) -> None: + if self._interrupt_after is not boundary: + return with self._engine.begin() as connection: assert_worker_role(connection) - return connection.execute( + recorded = connection.execute( text( """ - SELECT * - FROM public.context_worker_activate_file_replacement( + SELECT public.context_worker_record_file_import_interruption( :organization_id, :job_id, :service_principal_id, - :source_ref, :resource_ref, - :previous_revision_id, :replacement_revision_id, + :source_ref, :boundary, :lease_generation, :signing_key_version, :nonce, :issued_at, :expires_at ) """ @@ -492,15 +558,17 @@ def _replace( "job_id": claims.job_id, "service_principal_id": claims.service_principal_id, "source_ref": claims.source_ref, - "resource_ref": resource_ref, - "previous_revision_id": staged.previous_revision_id, - "replacement_revision_id": staged.replacement_revision_id, + "boundary": boundary.value, + "lease_generation": claims.lease_generation, "signing_key_version": claims.signing_key_version, "nonce": claims.nonce, "issued_at": claims.issued_at, "expires_at": claims.expires_at, }, - ).one_or_none() + ).scalar_one() + if recorded is not True: + raise _rejection(token) + raise FileImportInterrupted(boundary) def _fail( self, @@ -517,7 +585,8 @@ def _fail( """ SELECT public.context_worker_fail_file_import( :organization_id, :job_id, :service_principal_id, - :source_ref, :signing_key_version, :nonce, + :source_ref, :lease_generation, + :signing_key_version, :nonce, :issued_at, :expires_at ) """ @@ -527,6 +596,7 @@ def _fail( "job_id": claims.job_id, "service_principal_id": claims.service_principal_id, "source_ref": claims.source_ref, + "lease_generation": claims.lease_generation, "signing_key_version": claims.signing_key_version, "nonce": claims.nonce, "issued_at": claims.issued_at, diff --git a/engine/persistence/schema_security_manifest.yaml b/engine/persistence/schema_security_manifest.yaml index 9d15f63b..3bdba00b 100644 --- a/engine/persistence/schema_security_manifest.yaml +++ b/engine/persistence/schema_security_manifest.yaml @@ -1,5 +1,5 @@ { - "manifestVersion": "14.0.0", + "manifestVersion": "15.0.0", "controlOperations": [ { "name": "register_file_source", @@ -72,7 +72,9 @@ "directTableMutationAllowed": false, "databaseOwnedTime": true, "maxTtlSeconds": 3600, - "atomicWrites": ["file_import_job"] + "signedLeaseGeneration": true, + "reclaimableStatesAfterExpiry": ["leased", "running", "prepared", "ready"], + "atomicWrites": ["file_import_job", "file_import_job_event"] }, { "name": "redeem_file_import_lease", @@ -142,6 +144,31 @@ "file_revision_supersession", "file_import_job" ] }, + { + "name": "recover_file_publication", + "databaseFunctions": [ + "context_worker_redeem_file_import", + "context_worker_acquire_file_publication", + "context_worker_prepare_file_publication", + "context_worker_index_file_publication", + "context_worker_activate_recoverable_file_publication", + "context_worker_record_file_import_interruption" + ], + "role": "context_engine_worker", + "definerRole": "context_engine_worker_lease_definer", + "directTableMutationAllowed": false, + "durableBoundaries": ["acquired", "prepared", "ready", "completed"], + "leaseReclaim": "only an expired leased, running, prepared, or ready job can receive a higher lease generation; the old nonce can no longer redeem or mutate", + "idempotencyBinding": ["organization_id", "job_id", "source_id", "resource_ref", "revision_id", "content_identity_digest", "publication_payload_digest"], + "concurrentPublicationArbitration": "the Organization/Source/Resource ingestion guard serializes classification; a late identical job observes the winner as unchanged", + "activationInvariant": "authorization and complete Fragment/index evidence are revalidated before one active-pointer transaction", + "atomicWritesByBoundary": { + "acquired": ["file_resource_ingestion_guard", "file_acquisition_result", "file_publication_recovery", "file_import_job", "file_import_job_event"], + "prepared": ["context_resource", "context_revision", "file_revision_snapshot", "context_fragment", "revision_publication_event", "resource_access_policy", "membership_resource_field_right", "file_publication_recovery", "file_import_job", "file_import_job_event"], + "ready": ["exact_phrase_candidate", "revision_publication_event", "file_revision_replacement_plan", "file_publication_recovery", "file_import_job", "file_import_job_event"], + "completed": ["context_resource", "revision_publication_event", "file_revision_supersession", "file_publication_recovery", "file_import_job", "file_import_job_event"] + } + }, { "name": "fail_file_import", "databaseFunction": "context_worker_fail_file_import", @@ -717,9 +744,10 @@ } ] }, + "functionOnlyMutation": {"databaseFunctions": ["context_worker_prepare_file_publication", "context_worker_activate_recoverable_file_publication"], "definerRole": "context_engine_worker_lease_definer", "directTableMutationAllowed": false}, "permittedOperations": { "context_engine_runtime": ["SELECT"], - "context_engine_worker": [], + "context_engine_worker": ["EXECUTE context_worker_prepare_file_publication", "EXECUTE context_worker_activate_recoverable_file_publication"], "context_engine_worker_lease_definer": ["SELECT", "INSERT", "UPDATE"] }, "partitions": [], @@ -795,9 +823,10 @@ "events": ["UPDATE", "DELETE"], "sqlstate": "55000" }, + "functionOnlyMutation": {"databaseFunction": "context_worker_prepare_file_publication", "definerRole": "context_engine_worker_lease_definer", "directTableMutationAllowed": false}, "permittedOperations": { "context_engine_runtime": ["SELECT"], - "context_engine_worker": [], + "context_engine_worker": ["EXECUTE context_worker_prepare_file_publication"], "context_engine_worker_lease_definer": ["INSERT"] }, "partitions": [], @@ -996,11 +1025,12 @@ } ] }, + "functionOnlyMutation": {"databaseFunction": "context_worker_prepare_file_publication", "definerRole": "context_engine_worker_lease_definer", "directTableMutationAllowed": false}, "permittedOperations": { "context_engine_access_policy_definer": ["SELECT", "UPDATE"], "context_engine_control": ["EXECUTE change_resource_access"], "context_engine_runtime": ["SELECT"], - "context_engine_worker": [], + "context_engine_worker": ["EXECUTE context_worker_prepare_file_publication"], "context_engine_worker_lease_definer": ["SELECT", "INSERT"] }, "partitions": [], @@ -1750,9 +1780,10 @@ "events": ["UPDATE", "DELETE"], "sqlstate": "55000" }, + "functionOnlyMutation": {"databaseFunction": "context_worker_prepare_file_publication", "definerRole": "context_engine_worker_lease_definer", "directTableMutationAllowed": false}, "permittedOperations": { "context_engine_runtime": ["SELECT"], - "context_engine_worker": [], + "context_engine_worker": ["EXECUTE context_worker_prepare_file_publication"], "context_engine_worker_lease_definer": ["SELECT", "INSERT"] }, "partitions": [], @@ -1929,9 +1960,10 @@ } ] }, + "functionOnlyMutation": {"databaseFunction": "context_worker_prepare_file_publication", "definerRole": "context_engine_worker_lease_definer", "directTableMutationAllowed": false}, "permittedOperations": { "context_engine_runtime": ["SELECT"], - "context_engine_worker": [], + "context_engine_worker": ["EXECUTE context_worker_prepare_file_publication"], "context_engine_worker_lease_definer": ["SELECT", "INSERT"] }, "partitions": [], @@ -2504,8 +2536,8 @@ ] }, "immutableRows": {"trigger": "file_resource_ingestion_guard_immutable", "function": "context_content_reject_mutation", "events": ["UPDATE", "DELETE"], "sqlstate": "55000"}, - "functionOnlyMutation": {"databaseFunctions": ["context_worker_publish_file_import_v2", "context_worker_publish_structural_file_import_v2", "context_worker_stage_file_replacement", "context_worker_stage_structural_file_replacement"], "definerRole": "context_engine_worker_lease_definer", "directTableMutationAllowed": false}, - "permittedOperations": {"context_engine_control": [], "context_engine_runtime": [], "context_engine_worker": ["EXECUTE context_worker_publish_file_import_v2", "EXECUTE context_worker_publish_structural_file_import_v2", "EXECUTE context_worker_stage_file_replacement", "EXECUTE context_worker_stage_structural_file_replacement"], "context_engine_worker_lease_definer": ["SELECT", "INSERT", "UPDATE resource_ref"]}, + "functionOnlyMutation": {"databaseFunctions": ["context_worker_publish_file_import_v2", "context_worker_publish_structural_file_import_v2", "context_worker_stage_file_replacement", "context_worker_stage_structural_file_replacement", "context_worker_acquire_file_publication"], "definerRole": "context_engine_worker_lease_definer", "directTableMutationAllowed": false}, + "permittedOperations": {"context_engine_control": [], "context_engine_runtime": [], "context_engine_worker": ["EXECUTE context_worker_publish_file_import_v2", "EXECUTE context_worker_publish_structural_file_import_v2", "EXECUTE context_worker_stage_file_replacement", "EXECUTE context_worker_stage_structural_file_replacement", "EXECUTE context_worker_acquire_file_publication"], "context_engine_worker_lease_definer": ["SELECT", "INSERT", "UPDATE resource_ref"]}, "partitions": [], "securityInvariantIds": ["TENANT-OWNERSHIP-001", "TENANT-FK-002", "RLS-FAIL-CLOSED-003", "WORKER-LEASE-007"], "negativeTestIds": ["DB-001", "DB-004", "DB-008", "PG-FILE-IMPORT-023"] @@ -2541,8 +2573,8 @@ ] }, "immutableRows": {"trigger": "file_acquisition_result_immutable", "function": "context_content_reject_mutation", "events": ["UPDATE", "DELETE"], "sqlstate": "55000"}, - "functionOnlyMutation": {"databaseFunctions": ["context_worker_publish_file_import_v2", "context_worker_publish_structural_file_import_v2", "context_worker_stage_file_replacement", "context_worker_stage_structural_file_replacement"], "definerRole": "context_engine_worker_lease_definer", "directTableMutationAllowed": false}, - "permittedOperations": {"context_engine_control": [], "context_engine_runtime": [], "context_engine_worker": ["EXECUTE context_worker_publish_file_import_v2", "EXECUTE context_worker_publish_structural_file_import_v2", "EXECUTE context_worker_stage_file_replacement", "EXECUTE context_worker_stage_structural_file_replacement"], "context_engine_worker_lease_definer": ["SELECT", "INSERT"]}, + "functionOnlyMutation": {"databaseFunctions": ["context_worker_publish_file_import_v2", "context_worker_publish_structural_file_import_v2", "context_worker_stage_file_replacement", "context_worker_stage_structural_file_replacement", "context_worker_acquire_file_publication"], "definerRole": "context_engine_worker_lease_definer", "directTableMutationAllowed": false}, + "permittedOperations": {"context_engine_control": [], "context_engine_runtime": [], "context_engine_worker": ["EXECUTE context_worker_publish_file_import_v2", "EXECUTE context_worker_publish_structural_file_import_v2", "EXECUTE context_worker_stage_file_replacement", "EXECUTE context_worker_stage_structural_file_replacement", "EXECUTE context_worker_acquire_file_publication"], "context_engine_worker_lease_definer": ["SELECT", "INSERT"]}, "retention": {"sourceContent": "none", "reason": "fixed code plus organization-scoped digest only"}, "partitions": [], "securityInvariantIds": ["TENANT-OWNERSHIP-001", "TENANT-FK-002", "RLS-FAIL-CLOSED-003", "WORKER-LEASE-007", "TRACE-REDACTION-012"], @@ -2578,8 +2610,9 @@ {"name": "ck_file_import_job_worker_audience", "expression": "worker_audience = 'context-engine-worker'"}, {"name": "ck_file_import_job_actor_kind", "expression": "actor_kind = 'service'"}, {"name": "ck_file_import_job_operation", "expression": "operation = 'file.import'"}, - {"name": "ck_file_import_job_state", "expression": "state IN ('available', 'leased', 'running', 'failed', 'completed')"}, - {"name": "ck_file_import_job_state_consistency", "expression": "state transitions bind one lease nonce, exact time window, one completion lineage, and effect_count IN (0, 1)"} + {"name": "ck_file_import_job_state", "expression": "state IN ('available', 'leased', 'running', 'prepared', 'ready', 'failed', 'completed')"}, + {"name": "ck_file_import_job_state_consistency", "expression": "state transitions bind one current lease generation and nonce, exact time window, durable recovery identity, and effect_count IN (0, 1)"}, + {"name": "ck_file_import_job_recovery_from_state", "expression": "recovery_from_state IS NULL OR recovery_from_state IN ('running', 'prepared', 'ready')"} ], "rowLevelSecurity": { "enabled": true, @@ -2591,12 +2624,83 @@ {"name": "file_import_job_definer_update", "command": "UPDATE", "roles": ["context_engine_worker_lease_definer"], "using": "organization_id = NULLIF(current_setting('app.organization_id', true), '')::uuid AND job_id = NULLIF(current_setting('app.worker_job_id', true), '')::uuid AND workload = 'supply.file-import' AND worker_audience = 'context-engine-worker' AND operation = 'file.import'", "withCheck": "organization_id = NULLIF(current_setting('app.organization_id', true), '')::uuid AND job_id = NULLIF(current_setting('app.worker_job_id', true), '')::uuid AND workload = 'supply.file-import' AND worker_audience = 'context-engine-worker' AND operation = 'file.import'"} ] }, - "functionOnlyMutation": {"databaseFunctions": ["context_control_prepare_file_import", "context_worker_issue_file_import_lease", "context_worker_redeem_file_import", "context_worker_fail_file_import", "context_worker_publish_file_import_v2", "context_worker_publish_structural_file_import_v2"], "definerRole": "context_engine_worker_lease_definer", "directTableMutationAllowed": false}, - "permittedOperations": {"context_engine_control": ["EXECUTE context_control_prepare_file_import", "EXECUTE context_worker_issue_file_import_lease"], "context_engine_runtime": [], "context_engine_worker": ["EXECUTE context_worker_redeem_file_import", "EXECUTE context_worker_fail_file_import", "EXECUTE context_worker_publish_file_import_v2", "EXECUTE context_worker_publish_structural_file_import_v2"], "context_engine_worker_lease_definer": ["SELECT", "INSERT", "UPDATE"]}, + "functionOnlyMutation": {"databaseFunctions": ["context_control_prepare_file_import", "context_worker_issue_file_import_lease", "context_worker_redeem_file_import", "context_worker_fail_file_import", "context_worker_publish_file_import_v2", "context_worker_publish_structural_file_import_v2", "context_worker_stage_file_replacement", "context_worker_stage_structural_file_replacement", "context_worker_activate_file_replacement", "context_worker_acquire_file_publication", "context_worker_prepare_file_publication", "context_worker_index_file_publication", "context_worker_activate_recoverable_file_publication", "context_worker_record_file_import_interruption"], "definerRole": "context_engine_worker_lease_definer", "directTableMutationAllowed": false}, + "permittedOperations": {"context_engine_control": ["EXECUTE context_control_prepare_file_import", "EXECUTE context_worker_issue_file_import_lease"], "context_engine_runtime": [], "context_engine_worker": ["EXECUTE context_worker_redeem_file_import", "EXECUTE context_worker_fail_file_import", "EXECUTE context_worker_publish_file_import_v2", "EXECUTE context_worker_publish_structural_file_import_v2", "EXECUTE context_worker_stage_file_replacement", "EXECUTE context_worker_stage_structural_file_replacement", "EXECUTE context_worker_activate_file_replacement", "EXECUTE context_worker_acquire_file_publication", "EXECUTE context_worker_prepare_file_publication", "EXECUTE context_worker_index_file_publication", "EXECUTE context_worker_activate_recoverable_file_publication", "EXECUTE context_worker_record_file_import_interruption"], "context_engine_worker_lease_definer": ["SELECT", "INSERT", "UPDATE"]}, "partitions": [], "securityInvariantIds": ["TENANT-OWNERSHIP-001", "TENANT-FK-002", "RLS-FAIL-CLOSED-003", "WORKER-LEASE-007"], "negativeTestIds": ["DB-001", "DB-004", "DB-008", "JOB-001", "JOB-005", "WORKER-LEASE-007"] }, + { + "name": "file_publication_recovery", + "classification": "tenant_owned", + "nonOwnerEvidence": { + "evidenceId": "PG-FILE-RECOVERY-027", + "selector": {"table": "file_publication_recovery"} + }, + "purpose": "Mutable Organization/job checkpoint binding one File publication to one stable Resource and Revision identity across lease generations", + "organizationColumn": "organization_id", + "organizationInclusiveKeys": [ + {"name": "pk_file_publication_recovery", "kind": "primary_key", "columns": ["organization_id", "job_id"]}, + {"name": "uq_file_publication_recovery_revision", "kind": "unique", "columns": ["organization_id", "resource_ref", "revision_id"]} + ], + "foreignKeys": [ + {"name": "fk_file_publication_recovery_job_same_organization", "columns": ["organization_id", "job_id"], "references": {"table": "file_import_job", "columns": ["organization_id", "job_id"]}}, + {"name": "fk_file_publication_recovery_guard_same_organization", "columns": ["organization_id", "source_id", "resource_ref"], "references": {"table": "file_resource_ingestion_guard", "columns": ["organization_id", "source_id", "resource_ref"]}}, + {"name": "fk_file_publication_recovery_previous_same_organization", "columns": ["organization_id", "resource_ref", "previous_revision_id"], "references": {"table": "context_revision", "columns": ["organization_id", "resource_ref", "revision_id"]}} + ], + "checkConstraints": [ + {"name": "ck_file_publication_recovery_kind", "expression": "initial has no previous Revision; replacement has one distinct previous Revision"}, + {"name": "ck_file_publication_recovery_checkpoint", "expression": "checkpoint IN ('acquired', 'prepared', 'ready', 'completed')"}, + {"name": "ck_file_publication_recovery_digests", "expression": "content identity, content, compilation and exact publication-payload digests are SHA-256 hex"}, + {"name": "ck_file_publication_recovery_compiler", "expression": "only the accepted Markdown v1 and v2 compiler/config pairs are recoverable"} + ], + "rowLevelSecurity": {"enabled": true, "forced": true, "policies": [ + {"name": "file_publication_recovery_migrator_administration", "command": "ALL", "roles": ["context_engine_migrator"], "using": "true", "withCheck": "true"}, + {"name": "file_publication_recovery_file_import_definer_select", "command": "SELECT", "roles": ["context_engine_worker_lease_definer"], "using": "organization_id = NULLIF(current_setting('app.organization_id', true), '')::uuid"}, + {"name": "file_publication_recovery_file_import_definer_insert", "command": "INSERT", "roles": ["context_engine_worker_lease_definer"], "withCheck": "organization_id = NULLIF(current_setting('app.organization_id', true), '')::uuid"}, + {"name": "file_publication_recovery_file_import_definer_update", "command": "UPDATE", "roles": ["context_engine_worker_lease_definer"], "using": "organization_id = NULLIF(current_setting('app.organization_id', true), '')::uuid", "withCheck": "organization_id = NULLIF(current_setting('app.organization_id', true), '')::uuid"} + ]}, + "functionOnlyMutation": {"databaseFunctions": ["context_worker_acquire_file_publication", "context_worker_prepare_file_publication", "context_worker_index_file_publication", "context_worker_activate_recoverable_file_publication"], "definerRole": "context_engine_worker_lease_definer", "directTableMutationAllowed": false}, + "permittedOperations": {"context_engine_control": [], "context_engine_runtime": [], "context_engine_worker": ["EXECUTE context_worker_acquire_file_publication", "EXECUTE context_worker_prepare_file_publication", "EXECUTE context_worker_index_file_publication", "EXECUTE context_worker_activate_recoverable_file_publication"], "context_engine_worker_lease_definer": ["SELECT", "INSERT", "UPDATE"]}, + "retention": {"sourceContent": "none", "state": "retained with its publication job; cleanup authority is not active in Issue #27"}, + "partitions": [], + "securityInvariantIds": ["TENANT-OWNERSHIP-001", "TENANT-FK-002", "RLS-FAIL-CLOSED-003", "WORKER-LEASE-007", "TRACE-REDACTION-012"], + "negativeTestIds": ["DB-001", "DB-004", "DB-008", "PG-FILE-RECOVERY-027"] + }, + { + "name": "file_import_job_event", + "classification": "tenant_owned", + "nonOwnerEvidence": { + "evidenceId": "PG-FILE-RECOVERY-027", + "selector": {"table": "file_import_job_event"} + }, + "purpose": "Immutable ordered audit of durable publication boundaries, explicit interruption, lease reclaim and activation", + "organizationColumn": "organization_id", + "organizationInclusiveKeys": [ + {"name": "pk_file_import_job_event", "kind": "primary_key", "columns": ["organization_id", "job_id", "ordinal"]} + ], + "foreignKeys": [ + {"name": "fk_file_import_job_event_job_same_organization", "columns": ["organization_id", "job_id"], "references": {"table": "file_import_job", "columns": ["organization_id", "job_id"]}} + ], + "checkConstraints": [ + {"name": "ck_file_import_job_event_type", "expression": "event_type is acquired, prepared, indexed, interrupted, reclaimed, active or unchanged"}, + {"name": "ck_file_import_job_event_boundary", "expression": "boundary is acquired, prepared, indexed or active"}, + {"name": "ck_file_import_job_event_generation", "expression": "lease_generation > 0"}, + {"name": "ck_file_import_job_event_reason_digest", "expression": "interrupted, reclaimed and unchanged events require a SHA-256 reason digest; other events forbid it"} + ], + "rowLevelSecurity": {"enabled": true, "forced": true, "policies": [ + {"name": "file_import_job_event_migrator_administration", "command": "ALL", "roles": ["context_engine_migrator"], "using": "true", "withCheck": "true"}, + {"name": "file_import_job_event_file_import_definer_select", "command": "SELECT", "roles": ["context_engine_worker_lease_definer"], "using": "organization_id = NULLIF(current_setting('app.organization_id', true), '')::uuid"}, + {"name": "file_import_job_event_file_import_definer_insert", "command": "INSERT", "roles": ["context_engine_worker_lease_definer"], "withCheck": "organization_id = NULLIF(current_setting('app.organization_id', true), '')::uuid"} + ]}, + "immutableRows": {"trigger": "file_import_job_event_immutable", "function": "context_content_reject_mutation", "events": ["UPDATE", "DELETE"], "sqlstate": "55000"}, + "functionOnlyMutation": {"databaseFunctions": ["context_worker_issue_file_import_lease", "context_worker_acquire_file_publication", "context_worker_prepare_file_publication", "context_worker_index_file_publication", "context_worker_activate_recoverable_file_publication", "context_worker_record_file_import_interruption"], "definerRole": "context_engine_worker_lease_definer", "directTableMutationAllowed": false}, + "permittedOperations": {"context_engine_control": ["EXECUTE context_worker_issue_file_import_lease"], "context_engine_runtime": [], "context_engine_worker": ["EXECUTE context_worker_acquire_file_publication", "EXECUTE context_worker_prepare_file_publication", "EXECUTE context_worker_index_file_publication", "EXECUTE context_worker_activate_recoverable_file_publication", "EXECUTE context_worker_record_file_import_interruption"], "context_engine_worker_lease_definer": ["SELECT", "INSERT"]}, + "retention": {"sourceContent": "none", "reason": "fixed categories plus Organization/job/revision and digests only"}, + "partitions": [], + "securityInvariantIds": ["TENANT-OWNERSHIP-001", "TENANT-FK-002", "RLS-FAIL-CLOSED-003", "WORKER-LEASE-007", "TRACE-REDACTION-012"], + "negativeTestIds": ["DB-001", "DB-004", "DB-008", "PG-FILE-RECOVERY-027"] + }, { "name": "file_revision_snapshot", "classification": "tenant_owned", @@ -2637,7 +2741,8 @@ {"name": "file_revision_snapshot_file_noop_definer_select", "command": "SELECT", "roles": ["context_engine_worker_lease_definer"], "using": "organization_id = NULLIF(current_setting('app.organization_id', true), '')::uuid"} ]}, "immutableRows": {"trigger": "file_revision_snapshot_immutable", "function": "context_content_reject_mutation", "events": ["UPDATE", "DELETE"], "sqlstate": "55000"}, - "permittedOperations": {"context_engine_control": [], "context_engine_runtime": [], "context_engine_worker": ["EXECUTE context_worker_publish_file_import_v2", "EXECUTE context_worker_publish_structural_file_import_v2"], "context_engine_worker_lease_definer": ["SELECT", "INSERT"]}, + "functionOnlyMutation": {"databaseFunction": "context_worker_prepare_file_publication", "definerRole": "context_engine_worker_lease_definer", "directTableMutationAllowed": false}, + "permittedOperations": {"context_engine_control": [], "context_engine_runtime": [], "context_engine_worker": ["EXECUTE context_worker_publish_file_import_v2", "EXECUTE context_worker_publish_structural_file_import_v2", "EXECUTE context_worker_prepare_file_publication"], "context_engine_worker_lease_definer": ["SELECT", "INSERT"]}, "partitions": [], "securityInvariantIds": ["TENANT-OWNERSHIP-001", "TENANT-FK-002", "RLS-FAIL-CLOSED-003", "TRACE-REDACTION-012"], "negativeTestIds": ["DB-001", "DB-004", "DB-008", "WORKER-LEASE-007"] @@ -2668,7 +2773,8 @@ {"name": "revision_publication_event_current_user_actor", "command": "SELECT", "roles": ["context_engine_runtime"], "using": "organization_id = NULLIF(current_setting('app.organization_id', true), '')::uuid AND current_setting('app.actor_kind', true) = 'user' AND EXISTS (SELECT 1 FROM public.membership AS actor_membership WHERE actor_membership.organization_id = revision_publication_event.organization_id AND actor_membership.user_id = NULLIF(current_setting('app.user_id', true), '')::uuid AND actor_membership.membership_id = NULLIF(current_setting('app.membership_id', true), '')::uuid AND actor_membership.membership_version = NULLIF(current_setting('app.membership_version', true), '')::bigint AND actor_membership.status = 'active' AND actor_membership.valid_from <= NULLIF(current_setting('app.checked_at', true), '')::timestamptz AND (actor_membership.valid_until IS NULL OR actor_membership.valid_until > NULLIF(current_setting('app.checked_at', true), '')::timestamptz)) AND EXISTS (SELECT 1 FROM public.resource_access_policy AS access_policy WHERE access_policy.organization_id = revision_publication_event.organization_id AND access_policy.resource_ref = revision_publication_event.resource_ref AND access_policy.principal_ref = current_setting('app.principal_ref', true) AND access_policy.access_state = 'allowed')"} ]}, "immutableRows": {"trigger": "revision_publication_event_immutable", "function": "context_content_reject_mutation", "events": ["UPDATE", "DELETE"], "sqlstate": "55000"}, - "permittedOperations": {"context_engine_control": [], "context_engine_runtime": ["SELECT"], "context_engine_worker": ["EXECUTE context_worker_publish_file_import_v2", "EXECUTE context_worker_publish_structural_file_import_v2"], "context_engine_worker_lease_definer": ["SELECT", "INSERT"]}, + "functionOnlyMutation": {"databaseFunctions": ["context_worker_prepare_file_publication", "context_worker_index_file_publication", "context_worker_activate_recoverable_file_publication"], "definerRole": "context_engine_worker_lease_definer", "directTableMutationAllowed": false}, + "permittedOperations": {"context_engine_control": [], "context_engine_runtime": ["SELECT"], "context_engine_worker": ["EXECUTE context_worker_publish_file_import_v2", "EXECUTE context_worker_publish_structural_file_import_v2", "EXECUTE context_worker_prepare_file_publication", "EXECUTE context_worker_index_file_publication", "EXECUTE context_worker_activate_recoverable_file_publication"], "context_engine_worker_lease_definer": ["SELECT", "INSERT"]}, "partitions": [], "securityInvariantIds": ["TENANT-OWNERSHIP-001", "TENANT-FK-002", "RLS-FAIL-CLOSED-003", "TRACE-REDACTION-012"], "negativeTestIds": ["DB-001", "DB-004", "DB-008", "WORKER-LEASE-007"] @@ -2702,7 +2808,8 @@ ] }, "immutableRows": {"trigger": "exact_phrase_candidate_immutable", "function": "context_content_reject_mutation", "events": ["UPDATE", "DELETE"], "sqlstate": "55000"}, - "permittedOperations": {"context_engine_control": [], "context_engine_runtime": ["SELECT"], "context_engine_worker": ["EXECUTE context_worker_publish_file_import_v2", "EXECUTE context_worker_publish_structural_file_import_v2"], "context_engine_worker_lease_definer": ["SELECT", "INSERT"]}, + "functionOnlyMutation": {"databaseFunction": "context_worker_index_file_publication", "definerRole": "context_engine_worker_lease_definer", "directTableMutationAllowed": false}, + "permittedOperations": {"context_engine_control": [], "context_engine_runtime": ["SELECT"], "context_engine_worker": ["EXECUTE context_worker_publish_file_import_v2", "EXECUTE context_worker_publish_structural_file_import_v2", "EXECUTE context_worker_index_file_publication"], "context_engine_worker_lease_definer": ["SELECT", "INSERT"]}, "partitions": [], "securityInvariantIds": ["TENANT-OWNERSHIP-001", "TENANT-FK-002", "RLS-FAIL-CLOSED-003", "INDEX-NOT-AUTHORITY-005"], "negativeTestIds": ["DB-001", "DB-002", "DB-004", "DB-008", "DB-009", "DB-010", "PG-INDEX-NOT-AUTHORITY-005"] @@ -2738,8 +2845,8 @@ {"name": "file_revision_replacement_plan_file_import_definer_insert", "command": "INSERT", "roles": ["context_engine_worker_lease_definer"], "withCheck": "organization_id = NULLIF(current_setting('app.organization_id', true), '')::uuid"} ]}, "immutableRows": {"trigger": "file_revision_replacement_plan_immutable", "function": "context_content_reject_mutation", "events": ["UPDATE", "DELETE"], "sqlstate": "55000"}, - "functionOnlyMutation": {"databaseFunctions": ["context_worker_stage_file_replacement", "context_worker_stage_structural_file_replacement"], "definerRole": "context_engine_worker_lease_definer", "directTableMutationAllowed": false}, - "permittedOperations": {"context_engine_control": [], "context_engine_runtime": [], "context_engine_worker": ["EXECUTE context_worker_stage_file_replacement", "EXECUTE context_worker_stage_structural_file_replacement"], "context_engine_worker_lease_definer": ["SELECT", "INSERT"]}, + "functionOnlyMutation": {"databaseFunctions": ["context_worker_stage_file_replacement", "context_worker_stage_structural_file_replacement", "context_worker_index_file_publication"], "definerRole": "context_engine_worker_lease_definer", "directTableMutationAllowed": false}, + "permittedOperations": {"context_engine_control": [], "context_engine_runtime": [], "context_engine_worker": ["EXECUTE context_worker_stage_file_replacement", "EXECUTE context_worker_stage_structural_file_replacement", "EXECUTE context_worker_index_file_publication"], "context_engine_worker_lease_definer": ["SELECT", "INSERT"]}, "partitions": [], "securityInvariantIds": ["TENANT-OWNERSHIP-001", "TENANT-FK-002", "RLS-FAIL-CLOSED-003", "WORKER-LEASE-007"], "negativeTestIds": ["DB-001", "DB-004", "DB-008", "PG-FILE-REPLACEMENT-026"] @@ -2773,8 +2880,8 @@ {"name": "file_revision_supersession_file_import_definer_insert", "command": "INSERT", "roles": ["context_engine_worker_lease_definer"], "withCheck": "organization_id = NULLIF(current_setting('app.organization_id', true), '')::uuid"} ]}, "immutableRows": {"trigger": "file_revision_supersession_immutable", "function": "context_content_reject_mutation", "events": ["UPDATE", "DELETE"], "sqlstate": "55000"}, - "functionOnlyMutation": {"databaseFunction": "context_worker_activate_file_replacement", "definerRole": "context_engine_worker_lease_definer", "directTableMutationAllowed": false}, - "permittedOperations": {"context_engine_control": [], "context_engine_runtime": [], "context_engine_worker": ["EXECUTE context_worker_activate_file_replacement"], "context_engine_worker_lease_definer": ["SELECT", "INSERT"]}, + "functionOnlyMutation": {"databaseFunctions": ["context_worker_activate_file_replacement", "context_worker_activate_recoverable_file_publication"], "definerRole": "context_engine_worker_lease_definer", "directTableMutationAllowed": false}, + "permittedOperations": {"context_engine_control": [], "context_engine_runtime": [], "context_engine_worker": ["EXECUTE context_worker_activate_file_replacement", "EXECUTE context_worker_activate_recoverable_file_publication"], "context_engine_worker_lease_definer": ["SELECT", "INSERT"]}, "retention": {"supersededRevision": "retained_until_explicit_cleanup", "cleanupAuthority": "not active in Issue #26"}, "partitions": [], "securityInvariantIds": ["TENANT-OWNERSHIP-001", "TENANT-FK-002", "RLS-FAIL-CLOSED-003", "WORKER-LEASE-007"], diff --git a/engine/persistence/worker_jobs.py b/engine/persistence/worker_jobs.py index 89dcf040..120ae98a 100644 --- a/engine/persistence/worker_jobs.py +++ b/engine/persistence/worker_jobs.py @@ -252,7 +252,7 @@ def issue_file_import_lease( row = connection.execute( text( """ - SELECT issued_at, expires_at + SELECT issued_at, expires_at, lease_generation FROM public.context_worker_issue_file_import_lease( :organization_id, :job_id, :service_principal_id, :source_ref, @@ -287,6 +287,7 @@ def issue_file_import_lease( nonce=nonce, operation=FILE_IMPORT_WORKER_LEASE_OPERATION, source_ref=str(prepared.source_ref.value), + lease_generation=row.lease_generation, ) token = self._codec.mint(claims) return token diff --git a/engine/supply/jobs.py b/engine/supply/jobs.py index 0f5e78a8..e1816050 100644 --- a/engine/supply/jobs.py +++ b/engine/supply/jobs.py @@ -22,7 +22,7 @@ _ALGORITHM: Final = "HS256" _TOKEN_TYPE: Final = "CE-WorkerLease" _TOKEN_VERSION: Final = 1 -_FILE_IMPORT_TOKEN_VERSION: Final = 2 +_FILE_IMPORT_TOKEN_VERSION: Final = 3 _DOMAIN: Final = "context-engine.worker-lease" _MAX_KEY_VERSION: Final = (1 << 63) - 1 _MINIMUM_SECRET_BYTES: Final = 32 @@ -44,7 +44,9 @@ "workload", } ) -_FILE_IMPORT_CLAIM_FIELDS: Final = _CLAIM_FIELDS | frozenset({"source_ref"}) +_FILE_IMPORT_CLAIM_FIELDS: Final = _CLAIM_FIELDS | frozenset( + {"lease_generation", "source_ref"} +) def _require_key_version(value: object) -> int: @@ -100,6 +102,7 @@ class WorkerLeaseClaims: nonce: bytes = field(repr=False) operation: str = field(default=WORKER_LEASE_OPERATION, repr=False) source_ref: str | None = field(default=None, repr=False) + lease_generation: int | None = field(default=None, repr=False) actor_kind: Literal["service"] = field( default=WORKER_LEASE_ACTOR_KIND, init=False, repr=False ) @@ -125,10 +128,20 @@ def __post_init__(self) -> None: }: raise ValueError("WorkerLease operation must be closed") if self.operation == WORKER_LEASE_OPERATION: - if self.source_ref is not None: - raise ValueError("no-op WorkerLease cannot bind a source") + if self.source_ref is not None or self.lease_generation is not None: + raise ValueError( + "no-op WorkerLease cannot bind a source or lease generation" + ) else: _require_identifier("source_ref", self.source_ref, maximum_length=255) + if ( + type(self.lease_generation) is not int + or not 1 <= self.lease_generation <= _MAX_KEY_VERSION + ): + raise ValueError( + "File WorkerLease generation must be a positive signed " + "64-bit integer" + ) def __reduce__(self) -> NoReturn: raise TypeError("WorkerLeaseClaims are not serializable") @@ -284,6 +297,7 @@ def _claims_document(claims: WorkerLeaseClaims) -> dict[str, object]: } if claims.operation == FILE_IMPORT_WORKER_LEASE_OPERATION: document["source_ref"] = claims.source_ref + document["lease_generation"] = claims.lease_generation return document @@ -460,6 +474,11 @@ def _verify_signed(self, token: WorkerLeaseToken) -> WorkerLeaseClaims: if token_version == _FILE_IMPORT_TOKEN_VERSION else None ), + lease_generation=( + cast(int, document["lease_generation"]) + if token_version == _FILE_IMPORT_TOKEN_VERSION + else None + ), ) diff --git a/eval/catalogs/m0-security-evidence.yaml b/eval/catalogs/m0-security-evidence.yaml index b288f362..4cfe4a4d 100644 --- a/eval/catalogs/m0-security-evidence.yaml +++ b/eval/catalogs/m0-security-evidence.yaml @@ -51,6 +51,7 @@ {"id": "PG-FILE-SOURCE-FK-021", "layer": "postgres", "selector": "tests/integration/test_file_source_registration.py::test_source_version_is_immutable_and_active_pointer_stays_in_organization"}, {"id": "PG-FILE-IMPORT-023", "layer": "postgres", "selector": "tests/integration/test_file_import_tracer.py::test_registered_file_import_publishes_one_exact_authorized_http_package"}, {"id": "PG-FILE-REPLACEMENT-026", "layer": "postgres", "selector": "tests/integration/test_zz_file_revision_replacement.py::test_replacement_does_not_change_another_organization_resource"}, + {"id": "PG-FILE-RECOVERY-027", "layer": "postgres", "selector": "tests/integration/test_zz_file_publication_recovery.py::test_recovery_tables_deny_direct_access_to_every_nonowner_role"}, {"id": "RUNTIME-TENANT-FK-002", "layer": "runtime", "selector": "tests/integration/test_runtime_authorized_evidence_integration.py::test_real_postgres_http_delivers_only_exact_authorized_evidence_bidirectionally"}, {"id": "PROP-RLS-FAIL-CLOSED-003", "layer": "property", "selector": "tests/unit/test_schema_security_manifest.py::test_tenant_owned_manifest_entry_preserves_every_security_property"}, {"id": "PG-RLS-FAIL-CLOSED-003", "layer": "postgres", "selector": "tests/integration/test_organization_isolation.py::test_missing_tenant_context_is_fail_closed_for_every_operation"}, @@ -106,13 +107,13 @@ {"id": "FIXTURE-ACCEPT-012", "layer": "runtime", "selector": "tests/unit/test_ticket_audience_separation.py::test_accept_012_context_read_ticket_cannot_create_an_action_effect"} ], "invariantMappings": [ - {"invariantRef": "TENANT-OWNERSHIP-001", "evidenceRefs": {"property": ["PROP-TENANT-OWNERSHIP-001"], "postgres": ["PG-FILE-SOURCE-RLS-021", "PG-RLS-ALL-TENANT-TABLES", "PG-FILE-IMPORT-023", "PG-FILE-REPLACEMENT-026"], "runtime": ["RUNTIME-TENANT-OWNERSHIP-001"]}}, + {"invariantRef": "TENANT-OWNERSHIP-001", "evidenceRefs": {"property": ["PROP-TENANT-OWNERSHIP-001"], "postgres": ["PG-FILE-SOURCE-RLS-021", "PG-RLS-ALL-TENANT-TABLES", "PG-FILE-IMPORT-023", "PG-FILE-REPLACEMENT-026", "PG-FILE-RECOVERY-027"], "runtime": ["RUNTIME-TENANT-OWNERSHIP-001"]}}, {"invariantRef": "TENANT-FK-002", "evidenceRefs": {"property": ["PROP-TENANT-FK-002"], "postgres": ["PG-TENANT-FK-002", "PG-FILE-SOURCE-FK-021", "PG-FILE-IMPORT-023", "PG-FILE-REPLACEMENT-026"], "runtime": ["RUNTIME-TENANT-FK-002"]}}, {"invariantRef": "RLS-FAIL-CLOSED-003", "evidenceRefs": {"property": ["PROP-RLS-FAIL-CLOSED-003"], "postgres": ["PG-RLS-FAIL-CLOSED-003", "PG-RLS-ALL-TENANT-TABLES", "PG-FILE-IMPORT-023", "PG-FILE-REPLACEMENT-026"], "runtime": ["RUNTIME-RLS-FAIL-CLOSED-003"]}}, {"invariantRef": "SCOPE-INTERSECTION-004", "evidenceRefs": {"property": ["PROP-SCOPE-INTERSECTION-004"], "postgres": ["PG-SCOPE-INTERSECTION-004", "PG-FIELD-PROJECTION-RLS-048"], "runtime": ["RUNTIME-SCOPE-INTERSECTION-004"]}}, {"invariantRef": "INDEX-NOT-AUTHORITY-005", "evidenceRefs": {"property": ["PROP-INDEX-NOT-AUTHORITY-005"], "postgres": ["PG-INDEX-NOT-AUTHORITY-005", "PG-FILE-IMPORT-023", "PG-FILE-REPLACEMENT-026"], "runtime": ["RUNTIME-INDEX-NOT-AUTHORITY-005"]}}, {"invariantRef": "REVOCATION-006", "evidenceRefs": {"property": ["PROP-REVOCATION-006"], "postgres": ["PG-REVOCATION-006"], "runtime": ["RUNTIME-REVOCATION-006"]}}, - {"invariantRef": "WORKER-LEASE-007", "evidenceRefs": {"property": ["PROP-WORKER-LEASE-007"], "postgres": ["PG-WORKER-LEASE-007", "PG-FILE-IMPORT-023", "PG-FILE-REPLACEMENT-026"], "runtime": ["RUNTIME-WORKER-LEASE-007"]}}, + {"invariantRef": "WORKER-LEASE-007", "evidenceRefs": {"property": ["PROP-WORKER-LEASE-007"], "postgres": ["PG-WORKER-LEASE-007", "PG-FILE-IMPORT-023", "PG-FILE-REPLACEMENT-026", "PG-FILE-RECOVERY-027"], "runtime": ["RUNTIME-WORKER-LEASE-007"]}}, {"invariantRef": "TRANSPORT-UNTRUSTED-008", "evidenceRefs": {"property": ["PROP-TRANSPORT-UNTRUSTED-008"], "postgres": ["PG-TRANSPORT-UNTRUSTED-008"], "runtime": ["RUNTIME-TRANSPORT-UNTRUSTED-008"]}}, {"invariantRef": "NON-ENUMERATION-009", "evidenceRefs": {"property": ["PROP-NON-ENUMERATION-009"], "postgres": ["PG-NON-ENUMERATION-009"], "runtime": ["RUNTIME-NON-ENUMERATION-009"]}}, {"invariantRef": "CITATION-AUTH-010", "evidenceRefs": {"property": ["PROP-CITATION-AUTH-010"], "postgres": ["PG-CITATION-AUTH-010"], "runtime": ["RUNTIME-CITATION-AUTH-010"]}}, diff --git a/migrations/versions/20260723_0015_file_publication_recovery.py b/migrations/versions/20260723_0015_file_publication_recovery.py new file mode 100644 index 00000000..5b4414b0 --- /dev/null +++ b/migrations/versions/20260723_0015_file_publication_recovery.py @@ -0,0 +1,2352 @@ +"""Recover one File publication through durable idempotent boundaries. + +Revision ID: 20260723_0015 +Revises: 20260723_0014 +Create Date: 2026-07-23 +""" + +# ruff: noqa: E501 + +from collections.abc import Sequence + +import sqlalchemy as sa +from alembic import op +from sqlalchemy.dialects import postgresql + +revision: str = "20260723_0015" +down_revision: str | None = "20260723_0014" +branch_labels: str | Sequence[str] | None = None +depends_on: str | Sequence[str] | None = None + +_CONTROL = "context_engine_control" +_WORKER = "context_engine_worker" +_DEFINER = "context_engine_worker_lease_definer" +_MAX_BIGINT = 9223372036854775807 +_MAX_TTL = 3600 +_ISSUE_SIGNATURE = "(uuid, uuid, uuid, text, bigint, bytea, integer)" +_LEGACY_REDEEM_SIGNATURE = "(uuid, uuid, uuid, text, bigint, bytea, timestamp with time zone, timestamp with time zone)" +_LEGACY_FAIL_SIGNATURE = _LEGACY_REDEEM_SIGNATURE +_REDEEM_SIGNATURE = "(uuid, uuid, uuid, text, bigint, bigint, bytea, timestamp with time zone, timestamp with time zone)" +_FAIL_SIGNATURE = _REDEEM_SIGNATURE +_ACQUIRE_SIGNATURE = "(uuid, uuid, uuid, text, text, uuid, text, text, text, text, text, jsonb, jsonb, bigint, bigint, bytea, timestamp with time zone, timestamp with time zone)" +_STEP_SIGNATURE = "(uuid, uuid, uuid, text, text, uuid, text, jsonb, jsonb, bigint, bigint, bytea, timestamp with time zone, timestamp with time zone)" +_ACTIVATE_SIGNATURE = "(uuid, uuid, uuid, text, text, uuid, bigint, bigint, bytea, timestamp with time zone, timestamp with time zone)" +_INTERRUPT_SIGNATURE = "(uuid, uuid, uuid, text, text, bigint, bigint, bytea, timestamp with time zone, timestamp with time zone)" +_PUBLISH_V1_SIGNATURE = "(uuid, uuid, uuid, text, text, uuid, text, text, text, text, text, text, text, text, bigint, bytea, timestamp with time zone, timestamp with time zone)" +_PUBLISH_V2_SIGNATURE = "(uuid, uuid, uuid, text, text, uuid, text, text, text, text, text, jsonb, bigint, bytea, timestamp with time zone, timestamp with time zone)" +_STAGE_V1_SIGNATURE = _PUBLISH_V1_SIGNATURE +_STAGE_V2_SIGNATURE = _PUBLISH_V2_SIGNATURE +_LEGACY_ACTIVATE_SIGNATURE = "(uuid, uuid, uuid, text, text, uuid, uuid, bigint, bytea, timestamp with time zone, timestamp with time zone)" +_PUBLISH_V1_GENERATION_SIGNATURE = "(uuid, uuid, uuid, text, text, uuid, text, text, text, text, text, text, text, text, bigint, bigint, bytea, timestamp with time zone, timestamp with time zone)" +_PUBLISH_V2_GENERATION_SIGNATURE = "(uuid, uuid, uuid, text, text, uuid, text, text, text, text, text, jsonb, bigint, bigint, bytea, timestamp with time zone, timestamp with time zone)" +_STAGE_V1_GENERATION_SIGNATURE = _PUBLISH_V1_GENERATION_SIGNATURE +_STAGE_V2_GENERATION_SIGNATURE = _PUBLISH_V2_GENERATION_SIGNATURE +_LEGACY_ACTIVATE_GENERATION_SIGNATURE = "(uuid, uuid, uuid, text, text, uuid, uuid, bigint, bigint, bytea, timestamp with time zone, timestamp with time zone)" + + +def _tenant_table(table: str) -> None: + op.execute(f"ALTER TABLE {table} ENABLE ROW LEVEL SECURITY") + op.execute(f"ALTER TABLE {table} FORCE ROW LEVEL SECURITY") + op.execute( + f"CREATE POLICY {table}_migrator_administration ON {table} " + "FOR ALL TO context_engine_migrator USING (true) WITH CHECK (true)" + ) + tenant = ( + "organization_id = NULLIF(" + "current_setting('app.organization_id', true), ''" + ")::uuid" + ) + op.execute( + f"CREATE POLICY {table}_file_import_definer_select ON {table} " + f"FOR SELECT TO {_DEFINER} USING ({tenant})" + ) + op.execute( + f"CREATE POLICY {table}_file_import_definer_insert ON {table} " + f"FOR INSERT TO {_DEFINER} WITH CHECK ({tenant})" + ) + + +def _job_constraint() -> str: + lease = ( + "lease_generation > 0 AND signing_key_version > 0 AND " + "octet_length(lease_nonce_digest) = 32 AND " + "lease_issued_at IS NOT NULL AND lease_expires_at > lease_issued_at" + ) + no_lineage = ( + "resource_ref IS NULL AND revision_id IS NULL AND fragment_ref IS NULL" + ) + durable_identity = "resource_ref IS NOT NULL AND revision_id IS NOT NULL" + return ( + "(state = 'available' AND lease_generation = 0 AND " + "signing_key_version IS NULL AND lease_nonce_digest IS NULL AND " + "lease_issued_at IS NULL AND lease_expires_at IS NULL AND " + "lease_redeemed_at IS NULL AND recovery_from_state IS NULL AND " + "failed_at IS NULL AND completed_at IS NULL AND " + f"{no_lineage} AND effect_count = 0) OR " + f"(state = 'leased' AND {lease} AND lease_redeemed_at IS NULL AND " + "failed_at IS NULL AND completed_at IS NULL AND effect_count = 0 AND " + "((recovery_from_state IS NULL AND " + f"{no_lineage}) OR " + "(recovery_from_state = 'running' AND fragment_ref IS NULL) OR " + "(recovery_from_state IN ('prepared', 'ready') AND " + f"{durable_identity} AND fragment_ref IS NOT NULL))) OR " + f"(state = 'running' AND {lease} AND " + "lease_redeemed_at >= lease_issued_at AND recovery_from_state IS NULL " + "AND failed_at IS NULL AND completed_at IS NULL AND fragment_ref IS NULL " + "AND ((resource_ref IS NULL AND revision_id IS NULL) OR " + f"{durable_identity}) AND effect_count = 0) OR " + f"(state IN ('prepared', 'ready') AND {lease} AND " + "lease_redeemed_at >= lease_issued_at AND recovery_from_state IS NULL " + "AND failed_at IS NULL AND completed_at IS NULL AND " + f"{durable_identity} AND fragment_ref IS NOT NULL AND effect_count = 0) OR " + f"(state = 'failed' AND {lease} AND " + "lease_redeemed_at >= lease_issued_at AND recovery_from_state IS NULL " + "AND failed_at >= lease_redeemed_at AND completed_at IS NULL AND " + f"{no_lineage} AND effect_count = 0) OR " + f"(state = 'completed' AND {lease} AND " + "lease_redeemed_at >= lease_issued_at AND recovery_from_state IS NULL " + "AND failed_at IS NULL AND completed_at >= lease_redeemed_at AND " + f"{durable_identity} AND fragment_ref IS NOT NULL AND " + "effect_count IN (0, 1))" + ) + + +def upgrade() -> None: + """Add durable File publication checkpoints, reclaim, and audit.""" + + op.add_column( + "file_import_job", + sa.Column( + "lease_generation", + sa.BigInteger(), + nullable=False, + server_default=sa.text("0"), + ), + ) + op.add_column( + "file_import_job", + sa.Column("recovery_from_state", sa.Text(), nullable=True), + ) + # Rows created before this revision already consumed their original lease. + # Generation zero remains reserved for jobs that have never been issued. + op.execute( + "UPDATE file_import_job SET lease_generation = 1 " + "WHERE state <> 'available'" + ) + op.drop_constraint("ck_file_import_job_state", "file_import_job", type_="check") + op.create_check_constraint( + "ck_file_import_job_state", + "file_import_job", + "state IN ('available', 'leased', 'running', 'prepared', 'ready', 'failed', 'completed')", + ) + op.drop_constraint( + "ck_file_import_job_state_consistency", "file_import_job", type_="check" + ) + op.create_check_constraint( + "ck_file_import_job_state_consistency", + "file_import_job", + _job_constraint(), + ) + op.create_check_constraint( + "ck_file_import_job_recovery_from_state", + "file_import_job", + "recovery_from_state IS NULL OR recovery_from_state IN ('running', 'prepared', 'ready')", + ) + + op.create_table( + "file_publication_recovery", + sa.Column("organization_id", postgresql.UUID(as_uuid=True), nullable=False), + sa.Column("job_id", postgresql.UUID(as_uuid=True), nullable=False), + sa.Column("source_id", postgresql.UUID(as_uuid=True), nullable=False), + sa.Column("resource_ref", sa.Text(), nullable=False), + sa.Column("revision_id", postgresql.UUID(as_uuid=True), nullable=False), + sa.Column( + "previous_revision_id", postgresql.UUID(as_uuid=True), nullable=True + ), + sa.Column("publication_kind", sa.Text(), nullable=False), + sa.Column("checkpoint", sa.Text(), nullable=False), + sa.Column("content_identity_digest", sa.Text(), nullable=False), + sa.Column("content_hash", sa.Text(), nullable=False), + sa.Column("compilation_digest", sa.Text(), nullable=False), + sa.Column("publication_payload_digest", sa.Text(), nullable=False), + sa.Column("compiler_version", sa.Text(), nullable=False), + sa.Column("config_version", sa.Text(), nullable=False), + sa.Column("created_at", sa.DateTime(timezone=True), nullable=False), + sa.Column("updated_at", sa.DateTime(timezone=True), nullable=False), + sa.PrimaryKeyConstraint( + "organization_id", "job_id", name="pk_file_publication_recovery" + ), + sa.UniqueConstraint( + "organization_id", + "resource_ref", + "revision_id", + name="uq_file_publication_recovery_revision", + ), + sa.ForeignKeyConstraint( + ["organization_id", "job_id"], + ["file_import_job.organization_id", "file_import_job.job_id"], + name="fk_file_publication_recovery_job_same_organization", + ), + sa.ForeignKeyConstraint( + ["organization_id", "source_id", "resource_ref"], + [ + "file_resource_ingestion_guard.organization_id", + "file_resource_ingestion_guard.source_id", + "file_resource_ingestion_guard.resource_ref", + ], + name="fk_file_publication_recovery_guard_same_organization", + ), + sa.ForeignKeyConstraint( + ["organization_id", "resource_ref", "previous_revision_id"], + [ + "context_revision.organization_id", + "context_revision.resource_ref", + "context_revision.revision_id", + ], + name="fk_file_publication_recovery_previous_same_organization", + ), + sa.CheckConstraint( + "publication_kind IN ('initial', 'replacement') AND " + "((publication_kind = 'initial' AND previous_revision_id IS NULL) OR " + "(publication_kind = 'replacement' AND previous_revision_id IS NOT NULL " + "AND previous_revision_id <> revision_id))", + name="ck_file_publication_recovery_kind", + ), + sa.CheckConstraint( + "checkpoint IN ('acquired', 'prepared', 'ready', 'completed')", + name="ck_file_publication_recovery_checkpoint", + ), + sa.CheckConstraint( + "content_identity_digest ~ '^[0-9a-f]{64}$' AND " + "content_hash ~ '^[0-9a-f]{64}$' AND " + "compilation_digest ~ '^[0-9a-f]{64}$' AND " + "publication_payload_digest ~ '^[0-9a-f]{64}$'", + name="ck_file_publication_recovery_digests", + ), + sa.CheckConstraint( + "(compiler_version = 'context-engine-markdown-v1' AND config_version = 'markdown-config-v1') OR " + "(compiler_version = 'context-engine-markdown-v2' AND config_version = 'markdown-config-v2')", + name="ck_file_publication_recovery_compiler", + ), + ) + op.create_table( + "file_import_job_event", + sa.Column("organization_id", postgresql.UUID(as_uuid=True), nullable=False), + sa.Column("job_id", postgresql.UUID(as_uuid=True), nullable=False), + sa.Column("ordinal", sa.BigInteger(), nullable=False), + sa.Column("event_type", sa.Text(), nullable=False), + sa.Column("boundary", sa.Text(), nullable=False), + sa.Column("lease_generation", sa.BigInteger(), nullable=False), + sa.Column("state_at_event", sa.Text(), nullable=False), + sa.Column("revision_id", postgresql.UUID(as_uuid=True), nullable=True), + sa.Column("reason_digest", sa.Text(), nullable=True), + sa.Column("occurred_at", sa.DateTime(timezone=True), nullable=False), + sa.PrimaryKeyConstraint( + "organization_id", "job_id", "ordinal", name="pk_file_import_job_event" + ), + sa.ForeignKeyConstraint( + ["organization_id", "job_id"], + ["file_import_job.organization_id", "file_import_job.job_id"], + name="fk_file_import_job_event_job_same_organization", + ), + sa.CheckConstraint( + "event_type IN ('acquired', 'prepared', 'indexed', 'interrupted', 'reclaimed', 'active', 'unchanged')", + name="ck_file_import_job_event_type", + ), + sa.CheckConstraint( + "boundary IN ('acquired', 'prepared', 'indexed', 'active')", + name="ck_file_import_job_event_boundary", + ), + sa.CheckConstraint( + "lease_generation > 0", name="ck_file_import_job_event_generation" + ), + sa.CheckConstraint( + "((event_type IN ('interrupted', 'reclaimed', 'unchanged') " + "AND reason_digest ~ '^[0-9a-f]{64}$') OR " + "(event_type NOT IN ('interrupted', 'reclaimed', 'unchanged') " + "AND reason_digest IS NULL))", + name="ck_file_import_job_event_reason_digest", + ), + ) + for table in ("file_publication_recovery", "file_import_job_event"): + _tenant_table(table) + op.execute( + "CREATE TRIGGER file_import_job_event_immutable BEFORE UPDATE OR DELETE " + "ON file_import_job_event FOR EACH ROW EXECUTE FUNCTION context_content_reject_mutation()" + ) + op.execute( + f"CREATE POLICY file_publication_recovery_file_import_definer_update " + "ON file_publication_recovery FOR UPDATE " + f"TO {_DEFINER} USING (organization_id = NULLIF(current_setting('app.organization_id', true), '')::uuid) " + "WITH CHECK (organization_id = NULLIF(current_setting('app.organization_id', true), '')::uuid)" + ) + op.execute( + f"GRANT SELECT, INSERT, UPDATE ON file_publication_recovery TO {_DEFINER}" + ) + op.execute(f"GRANT SELECT, INSERT ON file_import_job_event TO {_DEFINER}") + op.execute( + f"GRANT UPDATE (lease_generation, recovery_from_state) ON file_import_job TO {_DEFINER}" + ) + _backfill_ready_replacement_recovery() + op.execute(f"GRANT CREATE ON SCHEMA public TO {_DEFINER}") + op.execute(f"SET LOCAL ROLE {_DEFINER}") + op.execute( + f"DROP FUNCTION public.context_worker_issue_file_import_lease{_ISSUE_SIGNATURE}" + ) + op.execute( + f"DROP FUNCTION public.context_worker_redeem_file_import{_LEGACY_REDEEM_SIGNATURE}" + ) + op.execute( + f"DROP FUNCTION public.context_worker_fail_file_import{_LEGACY_FAIL_SIGNATURE}" + ) + for name, signature in ( + ("context_worker_publish_file_import_v2", _PUBLISH_V1_SIGNATURE), + ("context_worker_publish_structural_file_import_v2", _PUBLISH_V2_SIGNATURE), + ("context_worker_stage_file_replacement", _STAGE_V1_SIGNATURE), + ("context_worker_stage_structural_file_replacement", _STAGE_V2_SIGNATURE), + ("context_worker_activate_file_replacement", _LEGACY_ACTIVATE_SIGNATURE), + ): + op.execute( + f"REVOKE EXECUTE ON FUNCTION public.{name}{signature} FROM {_WORKER}" + ) + + _create_issue_function() + _create_redeem_function() + _create_recovery_safe_fail_function() + _create_generation_gated_legacy_wrappers() + _create_acquire_function() + _create_prepare_function() + _create_index_function() + _create_activate_function() + _create_interrupt_function() + for name, signature in ( + ("context_worker_issue_file_import_lease", _ISSUE_SIGNATURE), + ("context_worker_redeem_file_import", _REDEEM_SIGNATURE), + ("context_worker_fail_file_import", _FAIL_SIGNATURE), + ("context_worker_publish_file_import_v2", _PUBLISH_V1_GENERATION_SIGNATURE), + ("context_worker_publish_structural_file_import_v2", _PUBLISH_V2_GENERATION_SIGNATURE), + ("context_worker_stage_file_replacement", _STAGE_V1_GENERATION_SIGNATURE), + ("context_worker_stage_structural_file_replacement", _STAGE_V2_GENERATION_SIGNATURE), + ("context_worker_activate_file_replacement", _LEGACY_ACTIVATE_GENERATION_SIGNATURE), + ("context_worker_acquire_file_publication", _ACQUIRE_SIGNATURE), + ("context_worker_prepare_file_publication", _STEP_SIGNATURE), + ("context_worker_index_file_publication", _STEP_SIGNATURE), + ("context_worker_activate_recoverable_file_publication", _ACTIVATE_SIGNATURE), + ("context_worker_record_file_import_interruption", _INTERRUPT_SIGNATURE), + ): + op.execute(f"REVOKE ALL ON FUNCTION public.{name}{signature} FROM PUBLIC") + op.execute(f"ALTER FUNCTION public.{name}{signature} OWNER TO {_DEFINER}") + op.execute( + f"GRANT EXECUTE ON FUNCTION public.context_worker_issue_file_import_lease{_ISSUE_SIGNATURE} TO {_CONTROL}" + ) + for name, signature in ( + ("context_worker_redeem_file_import", _REDEEM_SIGNATURE), + ("context_worker_fail_file_import", _FAIL_SIGNATURE), + ("context_worker_publish_file_import_v2", _PUBLISH_V1_GENERATION_SIGNATURE), + ("context_worker_publish_structural_file_import_v2", _PUBLISH_V2_GENERATION_SIGNATURE), + ("context_worker_stage_file_replacement", _STAGE_V1_GENERATION_SIGNATURE), + ("context_worker_stage_structural_file_replacement", _STAGE_V2_GENERATION_SIGNATURE), + ("context_worker_activate_file_replacement", _LEGACY_ACTIVATE_GENERATION_SIGNATURE), + ("context_worker_acquire_file_publication", _ACQUIRE_SIGNATURE), + ("context_worker_prepare_file_publication", _STEP_SIGNATURE), + ("context_worker_index_file_publication", _STEP_SIGNATURE), + ("context_worker_activate_recoverable_file_publication", _ACTIVATE_SIGNATURE), + ("context_worker_record_file_import_interruption", _INTERRUPT_SIGNATURE), + ): + op.execute(f"GRANT EXECUTE ON FUNCTION public.{name}{signature} TO {_WORKER}") + op.execute("RESET ROLE") + op.execute(f"REVOKE CREATE ON SCHEMA public FROM {_DEFINER}") + + +def _backfill_ready_replacement_recovery() -> None: + """Adopt Issue #26 ready replacement plans into the recovery protocol.""" + + op.execute( + """ + WITH ready AS ( + SELECT job.organization_id, job.job_id, job.source_id, + job.resource_ref, job.revision_id, + job.lease_generation, plan.previous_revision_id, + plan.content_identity_digest, plan.prepared_at, + snapshot.content_hash, snapshot.compilation_digest, + snapshot.compiler_version, snapshot.config_version, + snapshot.canonical_text, snapshot.compilation_document, + CASE + WHEN snapshot.compilation_document IS NULL THEN + jsonb_build_array(jsonb_build_object( + 'fragmentRef', 'fragment:paragraph:1', + 'contextualText', split_part( + snapshot.canonical_text, chr(10), 3 + ), + 'searchPhrases', jsonb_build_array(split_part( + snapshot.canonical_text, chr(10), 3 + )) + )) + ELSE ( + SELECT jsonb_agg(jsonb_build_object( + 'fragmentRef', item.fragment->'fragmentRef', + 'contextualText', item.fragment->'contextualText', + 'searchPhrases', item.fragment->'searchPhrases' + ) ORDER BY item.ordinal) + FROM jsonb_array_elements( + snapshot.compilation_document->'fragments' + ) WITH ORDINALITY AS item(fragment, ordinal) + ) + END AS artifact_document + FROM file_import_job AS job + JOIN file_revision_replacement_plan AS plan + ON plan.organization_id = job.organization_id + AND plan.job_id = job.job_id + AND plan.resource_ref = job.resource_ref + AND plan.replacement_revision_id = job.revision_id + JOIN file_revision_snapshot AS snapshot + ON snapshot.organization_id = plan.organization_id + AND snapshot.resource_ref = plan.resource_ref + AND snapshot.revision_id = plan.replacement_revision_id + WHERE job.state = 'ready' + ) + INSERT INTO file_publication_recovery ( + organization_id, job_id, source_id, resource_ref, revision_id, + previous_revision_id, publication_kind, checkpoint, + content_identity_digest, content_hash, compilation_digest, + publication_payload_digest, compiler_version, config_version, + created_at, updated_at + ) + SELECT organization_id, job_id, source_id, resource_ref, revision_id, + previous_revision_id, 'replacement', 'ready', + content_identity_digest, content_hash, compilation_digest, + encode(digest(convert_to(jsonb_build_object( + 'compilationDocument', compilation_document, + 'artifactDocument', artifact_document + )::text, 'UTF8'), 'sha256'), 'hex'), + compiler_version, config_version, prepared_at, prepared_at + FROM ready + """ + ) + op.execute( + """ + INSERT INTO file_import_job_event ( + organization_id, job_id, ordinal, event_type, boundary, + lease_generation, state_at_event, revision_id, + reason_digest, occurred_at + ) + SELECT recovery.organization_id, recovery.job_id, event.ordinal, + event.event_type, event.boundary, job.lease_generation, + event.state_at_event, recovery.revision_id, NULL, + recovery.created_at + FROM file_publication_recovery AS recovery + JOIN file_import_job AS job + ON job.organization_id = recovery.organization_id + AND job.job_id = recovery.job_id + CROSS JOIN (VALUES + (0::bigint, 'acquired'::text, 'acquired'::text, 'running'::text), + (1::bigint, 'prepared'::text, 'prepared'::text, 'prepared'::text), + (2::bigint, 'indexed'::text, 'indexed'::text, 'ready'::text) + ) AS event(ordinal, event_type, boundary, state_at_event) + WHERE recovery.checkpoint = 'ready' + """ + ) + + +def _create_generation_gated_legacy_wrappers() -> None: + """Keep pre-recovery evidence seams usable without a generation bypass.""" + + generation_guard = """ + PERFORM pg_catalog.set_config( + 'app.organization_id', requested_organization_id::text, true + ); + PERFORM pg_catalog.set_config( + 'app.worker_job_id', requested_job_id::text, true + ); + IF SESSION_USER <> 'context_engine_worker' + OR requested_lease_generation NOT BETWEEN 1 AND 9223372036854775807 + OR NOT EXISTS ( + SELECT 1 FROM public.file_import_job AS job + WHERE job.organization_id = requested_organization_id + AND job.job_id = requested_job_id + AND job.lease_generation = requested_lease_generation + ) + THEN RETURN; END IF; + """ + op.execute( + f""" + CREATE FUNCTION public.context_worker_publish_file_import_v2( + requested_organization_id uuid, requested_job_id uuid, + requested_service_principal_id uuid, requested_source_ref text, + requested_resource_ref text, requested_revision_id uuid, + requested_fragment_ref text, requested_canonical_text text, + requested_paragraph text, requested_content_hash text, + requested_compilation_digest text, requested_compiler_version text, + requested_config_version text, requested_phrase_digest text, + requested_lease_generation bigint, + requested_signing_key_version bigint, requested_nonce bytea, + requested_issued_at timestamptz, requested_expires_at timestamptz + ) RETURNS TABLE ( + effect_count smallint, outcome text, active_revision_id uuid, + fragment_refs text[], content_identity_digest text, + reason_digest text + ) LANGUAGE plpgsql SECURITY DEFINER + SET search_path = pg_catalog, pg_temp SET row_security = on + AS $function$ BEGIN + {generation_guard} + RETURN QUERY SELECT * + FROM public.context_worker_publish_file_import_v2( + requested_organization_id, requested_job_id, + requested_service_principal_id, requested_source_ref, + requested_resource_ref, requested_revision_id, + requested_fragment_ref, requested_canonical_text, + requested_paragraph, requested_content_hash, + requested_compilation_digest, requested_compiler_version, + requested_config_version, requested_phrase_digest, + requested_signing_key_version, requested_nonce, + requested_issued_at, requested_expires_at + ); + END; $function$ + """ + ) + op.execute( + f""" + CREATE FUNCTION public.context_worker_publish_structural_file_import_v2( + requested_organization_id uuid, requested_job_id uuid, + requested_service_principal_id uuid, requested_source_ref text, + requested_resource_ref text, requested_revision_id uuid, + requested_canonical_text text, requested_content_hash text, + requested_compilation_digest text, requested_compiler_version text, + requested_config_version text, requested_compilation_document jsonb, + requested_lease_generation bigint, + requested_signing_key_version bigint, requested_nonce bytea, + requested_issued_at timestamptz, requested_expires_at timestamptz + ) RETURNS TABLE ( + effect_count smallint, outcome text, active_revision_id uuid, + fragment_refs text[], content_identity_digest text, + reason_digest text + ) LANGUAGE plpgsql SECURITY DEFINER + SET search_path = pg_catalog, pg_temp SET row_security = on + AS $function$ BEGIN + {generation_guard} + RETURN QUERY SELECT * + FROM public.context_worker_publish_structural_file_import_v2( + requested_organization_id, requested_job_id, + requested_service_principal_id, requested_source_ref, + requested_resource_ref, requested_revision_id, + requested_canonical_text, requested_content_hash, + requested_compilation_digest, requested_compiler_version, + requested_config_version, requested_compilation_document, + requested_signing_key_version, requested_nonce, + requested_issued_at, requested_expires_at + ); + END; $function$ + """ + ) + op.execute( + f""" + CREATE FUNCTION public.context_worker_stage_file_replacement( + requested_organization_id uuid, requested_job_id uuid, + requested_service_principal_id uuid, requested_source_ref text, + requested_resource_ref text, requested_revision_id uuid, + requested_fragment_ref text, requested_canonical_text text, + requested_paragraph text, requested_content_hash text, + requested_compilation_digest text, requested_compiler_version text, + requested_config_version text, requested_phrase_digest text, + requested_lease_generation bigint, + requested_signing_key_version bigint, requested_nonce bytea, + requested_issued_at timestamptz, requested_expires_at timestamptz + ) RETURNS TABLE ( + previous_revision_id uuid, replacement_revision_id uuid, + fragment_refs text[], content_identity_digest text, + effect_count smallint, outcome text, active_revision_id uuid, + reason_digest text + ) LANGUAGE plpgsql SECURITY DEFINER + SET search_path = pg_catalog, pg_temp SET row_security = on + AS $function$ BEGIN + {generation_guard} + RETURN QUERY SELECT * + FROM public.context_worker_stage_file_replacement( + requested_organization_id, requested_job_id, + requested_service_principal_id, requested_source_ref, + requested_resource_ref, requested_revision_id, + requested_fragment_ref, requested_canonical_text, + requested_paragraph, requested_content_hash, + requested_compilation_digest, requested_compiler_version, + requested_config_version, requested_phrase_digest, + requested_signing_key_version, requested_nonce, + requested_issued_at, requested_expires_at + ); + END; $function$ + """ + ) + op.execute( + f""" + CREATE FUNCTION public.context_worker_stage_structural_file_replacement( + requested_organization_id uuid, requested_job_id uuid, + requested_service_principal_id uuid, requested_source_ref text, + requested_resource_ref text, requested_revision_id uuid, + requested_canonical_text text, requested_content_hash text, + requested_compilation_digest text, requested_compiler_version text, + requested_config_version text, requested_compilation_document jsonb, + requested_lease_generation bigint, + requested_signing_key_version bigint, requested_nonce bytea, + requested_issued_at timestamptz, requested_expires_at timestamptz + ) RETURNS TABLE ( + previous_revision_id uuid, replacement_revision_id uuid, + fragment_refs text[], content_identity_digest text, + effect_count smallint, outcome text, active_revision_id uuid, + reason_digest text + ) LANGUAGE plpgsql SECURITY DEFINER + SET search_path = pg_catalog, pg_temp SET row_security = on + AS $function$ BEGIN + {generation_guard} + RETURN QUERY SELECT * + FROM public.context_worker_stage_structural_file_replacement( + requested_organization_id, requested_job_id, + requested_service_principal_id, requested_source_ref, + requested_resource_ref, requested_revision_id, + requested_canonical_text, requested_content_hash, + requested_compilation_digest, requested_compiler_version, + requested_config_version, requested_compilation_document, + requested_signing_key_version, requested_nonce, + requested_issued_at, requested_expires_at + ); + END; $function$ + """ + ) + op.execute( + f""" + CREATE FUNCTION public.context_worker_activate_file_replacement( + requested_organization_id uuid, requested_job_id uuid, + requested_service_principal_id uuid, requested_source_ref text, + requested_resource_ref text, requested_previous_revision_id uuid, + requested_replacement_revision_id uuid, + requested_lease_generation bigint, + requested_signing_key_version bigint, requested_nonce bytea, + requested_issued_at timestamptz, requested_expires_at timestamptz + ) RETURNS TABLE ( + effect_count smallint, outcome text, active_revision_id uuid, + fragment_refs text[], content_identity_digest text, + reason_digest text + ) LANGUAGE plpgsql SECURITY DEFINER + SET search_path = pg_catalog, pg_temp SET row_security = on + AS $function$ BEGIN + {generation_guard} + RETURN QUERY SELECT * + FROM public.context_worker_activate_file_replacement( + requested_organization_id, requested_job_id, + requested_service_principal_id, requested_source_ref, + requested_resource_ref, requested_previous_revision_id, + requested_replacement_revision_id, + requested_signing_key_version, requested_nonce, + requested_issued_at, requested_expires_at + ); + END; $function$ + """ + ) + + +def _create_issue_function() -> None: + op.execute( + f""" + CREATE OR REPLACE FUNCTION public.context_worker_issue_file_import_lease( + requested_organization_id uuid, requested_job_id uuid, + requested_service_principal_id uuid, requested_source_ref text, + requested_signing_key_version bigint, requested_nonce bytea, + requested_ttl_seconds integer + ) RETURNS TABLE ( + issued_at timestamptz, expires_at timestamptz, + lease_generation bigint + ) + LANGUAGE plpgsql SECURITY DEFINER + SET search_path = pg_catalog, pg_temp SET row_security = on + AS $function$ + DECLARE + now_at timestamptz; + job_row public.file_import_job%ROWTYPE; + resume_state text; + next_generation bigint; + next_ordinal bigint; + BEGIN + IF SESSION_USER <> '{_CONTROL}' + OR requested_signing_key_version NOT BETWEEN 1 AND {_MAX_BIGINT} + OR pg_catalog.octet_length(requested_nonce) <> 32 + OR requested_ttl_seconds NOT BETWEEN 1 AND {_MAX_TTL} + THEN RETURN; END IF; + PERFORM pg_catalog.set_config( + 'app.organization_id', requested_organization_id::text, true + ); + PERFORM pg_catalog.set_config( + 'app.worker_job_id', requested_job_id::text, true + ); + now_at := pg_catalog.date_trunc( + 'second', pg_catalog.transaction_timestamp() + ); + SELECT * INTO job_row FROM public.file_import_job AS job + WHERE job.organization_id = requested_organization_id + AND job.job_id = requested_job_id + AND job.service_principal_id = requested_service_principal_id + AND job.source_id::text = requested_source_ref + AND ( + job.state = 'available' + OR ( + job.state IN ('leased', 'running', 'prepared', 'ready') + AND job.lease_expires_at <= now_at + ) + ) + AND EXISTS ( + SELECT 1 FROM public.service_principal AS principal + WHERE principal.organization_id = job.organization_id + AND principal.service_principal_id = job.service_principal_id + AND principal.workload = job.workload + AND principal.worker_audience = job.worker_audience + AND principal.operation = job.operation + AND principal.enabled IS TRUE + ) + FOR UPDATE; + IF job_row.job_id IS NULL THEN RETURN; END IF; + IF job_row.state = 'leased' THEN + resume_state := job_row.recovery_from_state; + ELSIF job_row.state = 'available' THEN + resume_state := NULL; + ELSE + resume_state := job_row.state; + END IF; + next_generation := job_row.lease_generation + 1; + IF job_row.lease_generation > 0 THEN + SELECT COALESCE(max(event.ordinal), -1) + 1 + INTO next_ordinal + FROM public.file_import_job_event AS event + WHERE event.organization_id = requested_organization_id + AND event.job_id = requested_job_id; + INSERT INTO public.file_import_job_event ( + organization_id, job_id, ordinal, event_type, boundary, + lease_generation, state_at_event, revision_id, + reason_digest, occurred_at + ) VALUES ( + requested_organization_id, requested_job_id, next_ordinal, + 'reclaimed', + CASE COALESCE(resume_state, 'running') + WHEN 'running' THEN 'acquired' + WHEN 'prepared' THEN 'prepared' + ELSE 'indexed' + END, + next_generation, job_row.state, job_row.revision_id, + encode(public.digest( + convert_to('context-engine.file-reclaim.v1', 'UTF8') + || decode('00', 'hex') + || uuid_send(requested_organization_id) + || uuid_send(requested_job_id) + || int8send(next_generation), + 'sha256' + ), 'hex'), now_at + ); + END IF; + UPDATE public.file_import_job AS job + SET state = 'leased', + lease_generation = next_generation, + recovery_from_state = resume_state, + signing_key_version = requested_signing_key_version, + lease_nonce_digest = public.digest(requested_nonce, 'sha256'), + lease_issued_at = now_at, + lease_expires_at = now_at + pg_catalog.make_interval( + secs => requested_ttl_seconds + ), + lease_redeemed_at = NULL + WHERE job.organization_id = requested_organization_id + AND job.job_id = requested_job_id + RETURNING job.lease_issued_at, job.lease_expires_at, + job.lease_generation + INTO issued_at, expires_at, lease_generation; + IF issued_at IS NOT NULL THEN RETURN NEXT; END IF; + END; $function$ + """ + ) + + +def _create_redeem_function() -> None: + op.execute( + f""" + CREATE OR REPLACE FUNCTION public.context_worker_redeem_file_import( + requested_organization_id uuid, requested_job_id uuid, + requested_service_principal_id uuid, requested_source_ref text, + requested_lease_generation bigint, + requested_signing_key_version bigint, requested_nonce bytea, + requested_issued_at timestamptz, requested_expires_at timestamptz + ) RETURNS TABLE ( + source_ref text, root_ref text, relative_path text, + audience_principal_ref text, audience_membership_id uuid, + audience_membership_version bigint, acquisition_id uuid + ) + LANGUAGE plpgsql SECURITY DEFINER + SET search_path = pg_catalog, pg_temp SET row_security = on + AS $function$ + DECLARE redeemed_at timestamptz; + BEGIN + IF SESSION_USER <> '{_WORKER}' THEN RETURN; END IF; + PERFORM pg_catalog.set_config( + 'app.organization_id', requested_organization_id::text, true + ); + PERFORM pg_catalog.set_config( + 'app.worker_job_id', requested_job_id::text, true + ); + redeemed_at := pg_catalog.statement_timestamp(); + UPDATE public.file_import_job AS job + SET state = COALESCE(job.recovery_from_state, 'running'), + recovery_from_state = NULL, + lease_redeemed_at = redeemed_at + WHERE job.organization_id = requested_organization_id + AND job.job_id = requested_job_id + AND job.service_principal_id = requested_service_principal_id + AND job.source_id::text = requested_source_ref + AND job.state = 'leased' + AND job.lease_generation = requested_lease_generation + AND job.signing_key_version = requested_signing_key_version + AND job.lease_nonce_digest = public.digest(requested_nonce, 'sha256') + AND job.lease_issued_at = requested_issued_at + AND job.lease_expires_at = requested_expires_at + AND redeemed_at >= job.lease_issued_at + AND redeemed_at < job.lease_expires_at + AND EXISTS ( + SELECT 1 FROM public.service_principal AS principal + WHERE principal.organization_id = job.organization_id + AND principal.service_principal_id = job.service_principal_id + AND principal.workload = job.workload + AND principal.worker_audience = job.worker_audience + AND principal.operation = job.operation + AND principal.enabled IS TRUE + ); + IF NOT FOUND THEN RETURN; END IF; + RETURN QUERY + SELECT job.source_id::text, version.root_ref, + acquisition.relative_path, + acquisition.audience_principal_ref, + acquisition.audience_membership_id, + acquisition.audience_membership_version, + acquisition.acquisition_id + FROM public.file_import_job AS job + JOIN public.file_acquisition AS acquisition + ON acquisition.organization_id = job.organization_id + AND acquisition.acquisition_id = job.acquisition_id + JOIN public.source_version AS version + ON version.organization_id = acquisition.organization_id + AND version.source_id = acquisition.source_id + AND version.version_id = acquisition.source_version_id + WHERE job.organization_id = requested_organization_id + AND job.job_id = requested_job_id; + END; $function$ + """ + ) + + +def _create_recovery_safe_fail_function() -> None: + op.execute( + f""" + CREATE OR REPLACE FUNCTION public.context_worker_fail_file_import( + requested_organization_id uuid, requested_job_id uuid, + requested_service_principal_id uuid, requested_source_ref text, + requested_lease_generation bigint, + requested_signing_key_version bigint, requested_nonce bytea, + requested_issued_at timestamptz, + requested_expires_at timestamptz + ) RETURNS boolean + LANGUAGE plpgsql SECURITY DEFINER + SET search_path = pg_catalog, pg_temp SET row_security = on + AS $function$ + DECLARE changed boolean := false; failed_now timestamptz; + BEGIN + IF SESSION_USER <> '{_WORKER}' THEN RETURN false; END IF; + PERFORM pg_catalog.set_config( + 'app.organization_id', requested_organization_id::text, true + ); + PERFORM pg_catalog.set_config( + 'app.worker_job_id', requested_job_id::text, true + ); + failed_now := pg_catalog.statement_timestamp(); + UPDATE public.file_import_job AS job + SET state = 'failed', failed_at = failed_now + WHERE job.organization_id = requested_organization_id + AND job.job_id = requested_job_id + AND job.state = 'running' + AND job.lease_generation = requested_lease_generation + AND job.resource_ref IS NULL + AND job.revision_id IS NULL + AND job.fragment_ref IS NULL + AND NOT EXISTS ( + SELECT 1 FROM public.file_publication_recovery AS recovery + WHERE recovery.organization_id = job.organization_id + AND recovery.job_id = job.job_id + ) + AND job.service_principal_id = requested_service_principal_id + AND job.source_id::text = requested_source_ref + AND job.signing_key_version = requested_signing_key_version + AND job.lease_nonce_digest = public.digest(requested_nonce, 'sha256') + AND job.lease_issued_at = requested_issued_at + AND job.lease_expires_at = requested_expires_at + AND failed_now >= job.lease_issued_at + AND failed_now < job.lease_expires_at + AND EXISTS ( + SELECT 1 FROM public.service_principal AS principal + WHERE principal.organization_id = job.organization_id + AND principal.service_principal_id = job.service_principal_id + AND principal.workload = job.workload + AND principal.worker_audience = job.worker_audience + AND principal.operation = job.operation + AND principal.enabled IS TRUE + ); + changed := FOUND; + RETURN changed; + END; $function$ + """ + ) + + +def _artifact_validation() -> str: + return r""" + jsonb_typeof(requested_artifact_document) = 'array' + AND jsonb_array_length(requested_artifact_document) BETWEEN 1 AND 4096 + AND NOT EXISTS ( + SELECT 1 FROM jsonb_array_elements( + requested_artifact_document + ) AS item(fragment) + WHERE jsonb_typeof(item.fragment) IS DISTINCT FROM 'object' + OR COALESCE(btrim(item.fragment->>'fragmentRef') = '', true) + OR COALESCE(btrim(item.fragment->>'contextualText') = '', true) + OR jsonb_typeof(item.fragment->'searchPhrases') + IS DISTINCT FROM 'array' + OR jsonb_array_length(item.fragment->'searchPhrases') + NOT BETWEEN 1 AND 4096 + OR EXISTS ( + SELECT 1 FROM jsonb_array_elements_text( + item.fragment->'searchPhrases' + ) AS phrase(value) + WHERE btrim(phrase.value) = '' + ) + ) + AND ( + SELECT count(DISTINCT item.fragment->>'fragmentRef') + FROM jsonb_array_elements(requested_artifact_document) + AS item(fragment) + ) = jsonb_array_length(requested_artifact_document) + """ + + +def _publication_artifact_validation() -> str: + """Bind the compact write artifact to the accepted compiler contract.""" + + return r""" + ( + requested_compiler_version = 'context-engine-markdown-v1' + AND requested_config_version = 'markdown-config-v1' + AND requested_compilation_document IS NULL + AND requested_artifact_document = jsonb_build_array( + jsonb_build_object( + 'fragmentRef', 'fragment:paragraph:1', + 'contextualText', split_part( + requested_canonical_text, chr(10), 3 + ), + 'searchPhrases', jsonb_build_array(split_part( + requested_canonical_text, chr(10), 3 + )) + ) + ) + ) OR ( + requested_compiler_version = 'context-engine-markdown-v2' + AND requested_config_version = 'markdown-config-v2' + AND jsonb_typeof(requested_compilation_document) = 'object' + AND requested_compilation_document->>'canonicalText' + IS NOT DISTINCT FROM requested_canonical_text + AND requested_compilation_document->>'contentHash' + IS NOT DISTINCT FROM requested_content_hash + AND requested_compilation_document->>'compilationDigest' + IS NOT DISTINCT FROM requested_compilation_digest + AND requested_compilation_document#>>'{provenance,compilerVersion}' + IS NOT DISTINCT FROM requested_compiler_version + AND requested_compilation_document#>>'{provenance,configVersion}' + IS NOT DISTINCT FROM requested_config_version + AND requested_compilation_document#>>'{provenance,canonicalizationProfile}' + IS NOT DISTINCT FROM 'markdown-structural-units-v2' + AND requested_compilation_document#>>'{provenance,compilationDigestProfile}' + IS NOT DISTINCT FROM 'rfc8785-sha256-v2' + AND jsonb_typeof(requested_compilation_document->'sections') = 'array' + AND jsonb_typeof(requested_compilation_document->'fragments') = 'array' + AND jsonb_array_length(requested_compilation_document->'fragments') + BETWEEN 1 AND 4096 + AND jsonb_array_length(requested_compilation_document->'sections') + = jsonb_array_length(requested_compilation_document->'fragments') + AND NOT EXISTS ( + SELECT 1 + FROM jsonb_array_elements( + requested_compilation_document->'fragments' + ) WITH ORDINALITY AS item(fragment, source_ordinal) + WHERE jsonb_typeof(item.fragment) IS DISTINCT FROM 'object' + OR COALESCE( + item.fragment->>'fragmentRef' + !~ '^fragment:(heading|paragraph|list|fenced_code|table):[1-9][0-9]*$', + true + ) + OR COALESCE( + item.fragment->>'kind' + NOT IN ('heading', 'paragraph', 'list', 'fenced_code', 'table'), + true + ) + OR CASE + WHEN jsonb_typeof(item.fragment->'path') = 'array' + THEN jsonb_array_length(item.fragment->'path') < 2 + OR EXISTS ( + SELECT 1 FROM jsonb_array_elements_text( + item.fragment->'path' + ) AS path(segment) + WHERE btrim(path.segment) = '' + OR path.segment <> btrim(path.segment) + ) + ELSE true + END + OR jsonb_typeof(item.fragment->'position') + IS DISTINCT FROM 'object' + OR COALESCE(item.fragment#>>'{position,start,line}' !~ '^[1-9][0-9]*$', true) + OR COALESCE(item.fragment#>>'{position,start,column}' !~ '^[1-9][0-9]*$', true) + OR COALESCE(item.fragment#>>'{position,start,byteOffset}' !~ '^[0-9]+$', true) + OR COALESCE(item.fragment#>>'{position,end,line}' !~ '^[1-9][0-9]*$', true) + OR COALESCE(item.fragment#>>'{position,end,column}' !~ '^[1-9][0-9]*$', true) + OR COALESCE(item.fragment#>>'{position,end,byteOffset}' !~ '^[1-9][0-9]*$', true) + OR CASE + WHEN item.fragment#>>'{position,start,byteOffset}' ~ '^[0-9]+$' + AND item.fragment#>>'{position,end,byteOffset}' ~ '^[1-9][0-9]*$' + THEN (item.fragment#>>'{position,start,byteOffset}')::numeric + >= (item.fragment#>>'{position,end,byteOffset}')::numeric + ELSE true + END + OR COALESCE( + translate( + item.fragment->>'sourceText', + U&'\0009\000A\000B\000C\000D\001C\001D\001E\001F\0020\0085\00A0\1680\2000\2001\2002\2003\2004\2005\2006\2007\2008\2009\200A\2028\2029\202F\205F\3000', + '' + ) = '', + true + ) + OR COALESCE( + translate( + item.fragment->>'contextualText', + U&'\0009\000A\000B\000C\000D\001C\001D\001E\001F\0020\0085\00A0\1680\2000\2001\2002\2003\2004\2005\2006\2007\2008\2009\200A\2028\2029\202F\205F\3000', + '' + ) = '', + true + ) + OR CASE + WHEN jsonb_typeof(item.fragment->'searchPhrases') = 'array' + THEN jsonb_array_length(item.fragment->'searchPhrases') + NOT BETWEEN 1 AND 4096 + OR EXISTS ( + SELECT 1 FROM jsonb_array_elements_text( + item.fragment->'searchPhrases' + ) AS phrase(value) + WHERE translate( + phrase.value, + U&'\0009\000A\000B\000C\000D\001C\001D\001E\001F\0020\0085\00A0\1680\2000\2001\2002\2003\2004\2005\2006\2007\2008\2009\200A\2028\2029\202F\205F\3000', + '' + ) = '' + ) + OR ( + SELECT count(DISTINCT phrase.value) + FROM jsonb_array_elements_text( + item.fragment->'searchPhrases' + ) AS phrase(value) + ) <> jsonb_array_length( + item.fragment->'searchPhrases' + ) + ELSE true + END + ) + AND NOT EXISTS ( + SELECT 1 + FROM jsonb_array_elements( + requested_compilation_document->'fragments' + ) WITH ORDINALITY AS fragment_item(fragment, ordinal) + JOIN jsonb_array_elements( + requested_compilation_document->'sections' + ) WITH ORDINALITY AS section_item(section, ordinal) + USING (ordinal) + WHERE fragment_item.fragment->'kind' + IS DISTINCT FROM section_item.section->'kind' + OR fragment_item.fragment->'path' + IS DISTINCT FROM section_item.section->'path' + OR fragment_item.fragment->'position' + IS DISTINCT FROM section_item.section->'position' + ) + AND ( + SELECT count(DISTINCT item.fragment->>'fragmentRef') + FROM jsonb_array_elements( + requested_compilation_document->'fragments' + ) AS item(fragment) + ) = jsonb_array_length( + requested_compilation_document->'fragments' + ) + AND requested_artifact_document = ( + SELECT jsonb_agg( + jsonb_build_object( + 'fragmentRef', item.fragment->'fragmentRef', + 'contextualText', item.fragment->'contextualText', + 'searchPhrases', item.fragment->'searchPhrases' + ) ORDER BY item.ordinal + ) + FROM jsonb_array_elements( + requested_compilation_document->'fragments' + ) WITH ORDINALITY AS item(fragment, ordinal) + ) + ) + """ + + +def _publication_payload_digest() -> str: + return """ + encode(public.digest(convert_to( + jsonb_build_object( + 'compilationDocument', requested_compilation_document, + 'artifactDocument', requested_artifact_document + )::text, + 'UTF8' + ), 'sha256'), 'hex') + """ + + +def _create_acquire_function() -> None: + artifact_validation = _artifact_validation() + publication_validation = _publication_artifact_validation() + payload_digest = _publication_payload_digest() + op.execute( + f""" + CREATE FUNCTION public.context_worker_acquire_file_publication( + requested_organization_id uuid, requested_job_id uuid, + requested_service_principal_id uuid, requested_source_ref text, + requested_resource_ref text, requested_revision_id uuid, + requested_canonical_text text, requested_content_hash text, + requested_compilation_digest text, requested_compiler_version text, + requested_config_version text, + requested_compilation_document jsonb, + requested_artifact_document jsonb, + requested_lease_generation bigint, + requested_signing_key_version bigint, requested_nonce bytea, + requested_issued_at timestamptz, requested_expires_at timestamptz + ) RETURNS TABLE ( + checkpoint text, publication_kind text, stable_revision_id uuid, + previous_revision_id uuid, fragment_refs text[], + content_identity_digest text, effect_count smallint, outcome text, + active_revision_id uuid, reason_digest text + ) + LANGUAGE plpgsql SECURITY DEFINER + SET search_path = pg_catalog, pg_temp SET row_security = on + AS $function$ + DECLARE + job_row public.file_import_job%ROWTYPE; + recovery_row public.file_publication_recovery%ROWTYPE; + decision record; + old_revision uuid; + active_fragments text[]; + now_at timestamptz; + next_ordinal bigint; + BEGIN + IF SESSION_USER <> '{_WORKER}' + OR requested_revision_id IS NULL + OR btrim(requested_resource_ref) = '' + OR requested_content_hash !~ '^[0-9a-f]{{64}}$' + OR requested_compilation_digest !~ '^[0-9a-f]{{64}}$' + OR encode(public.digest( + convert_to(requested_canonical_text, 'UTF8'), 'sha256' + ), 'hex') <> requested_content_hash + OR NOT ({artifact_validation}) + OR NOT ({publication_validation}) + THEN RETURN; END IF; + PERFORM pg_catalog.set_config( + 'app.organization_id', requested_organization_id::text, true + ); + PERFORM pg_catalog.set_config( + 'app.worker_job_id', requested_job_id::text, true + ); + now_at := pg_catalog.statement_timestamp(); + SELECT * INTO job_row FROM public.file_import_job AS job + WHERE job.organization_id = requested_organization_id + AND job.job_id = requested_job_id + AND job.service_principal_id = requested_service_principal_id + AND job.source_id::text = requested_source_ref + AND job.state IN ('running', 'prepared', 'ready') + AND job.lease_generation = requested_lease_generation + AND job.signing_key_version = requested_signing_key_version + AND job.lease_nonce_digest = public.digest(requested_nonce, 'sha256') + AND job.lease_issued_at = requested_issued_at + AND job.lease_expires_at = requested_expires_at + AND now_at < job.lease_expires_at + AND EXISTS ( + SELECT 1 FROM public.service_principal AS principal + WHERE principal.organization_id = job.organization_id + AND principal.service_principal_id = job.service_principal_id + AND principal.workload = job.workload + AND principal.worker_audience = job.worker_audience + AND principal.operation = job.operation + AND principal.enabled IS TRUE + ) + FOR UPDATE; + IF job_row.job_id IS NULL THEN RETURN; END IF; + + SELECT * INTO recovery_row + FROM public.file_publication_recovery AS recovery + WHERE recovery.organization_id = requested_organization_id + AND recovery.job_id = requested_job_id; + IF recovery_row.job_id IS NOT NULL THEN + IF recovery_row.source_id::text <> requested_source_ref + OR recovery_row.resource_ref <> requested_resource_ref + OR recovery_row.content_hash <> requested_content_hash + OR recovery_row.compilation_digest <> + requested_compilation_digest + OR recovery_row.compiler_version <> + requested_compiler_version + OR recovery_row.config_version <> requested_config_version + OR recovery_row.publication_payload_digest <> ({payload_digest}) + OR job_row.resource_ref <> recovery_row.resource_ref + OR job_row.revision_id <> recovery_row.revision_id + THEN RETURN; END IF; + IF job_row.state = 'running' + AND recovery_row.checkpoint <> 'acquired' + THEN RETURN; + ELSIF job_row.state = 'prepared' + AND recovery_row.checkpoint <> 'prepared' + THEN RETURN; + ELSIF job_row.state = 'ready' + AND recovery_row.checkpoint <> 'ready' + THEN RETURN; + END IF; + SELECT array_agg(fragment.fragment_ref ORDER BY fragment.ordinal) + INTO active_fragments + FROM public.context_fragment AS fragment + WHERE fragment.organization_id = requested_organization_id + AND fragment.resource_ref = recovery_row.resource_ref + AND fragment.revision_id = recovery_row.revision_id; + RETURN QUERY SELECT recovery_row.checkpoint, + recovery_row.publication_kind, recovery_row.revision_id, + recovery_row.previous_revision_id, active_fragments, + recovery_row.content_identity_digest, NULL::smallint, + NULL::text, NULL::uuid, NULL::text; + RETURN; + END IF; + IF job_row.state <> 'running' + OR job_row.resource_ref IS NOT NULL + OR job_row.revision_id IS NOT NULL + THEN RETURN; END IF; + + INSERT INTO public.file_resource_ingestion_guard ( + organization_id, source_id, resource_ref, created_at + ) VALUES ( + requested_organization_id, job_row.source_id, + requested_resource_ref, now_at + ) ON CONFLICT (organization_id, resource_ref) DO NOTHING; + PERFORM 1 FROM public.file_resource_ingestion_guard AS guard + WHERE guard.organization_id = requested_organization_id + AND guard.source_id = job_row.source_id + AND guard.resource_ref = requested_resource_ref + FOR UPDATE; + IF NOT FOUND THEN RETURN; END IF; + IF EXISTS ( + SELECT 1 FROM public.file_publication_recovery AS recovery + WHERE recovery.organization_id = requested_organization_id + AND recovery.resource_ref = requested_resource_ref + AND recovery.job_id <> requested_job_id + AND recovery.checkpoint <> 'completed' + ) THEN + RETURN QUERY SELECT 'contended'::text, NULL::text, + NULL::uuid, NULL::uuid, NULL::text[], NULL::text, + NULL::smallint, NULL::text, NULL::uuid, NULL::text; + RETURN; + END IF; + + SELECT * INTO decision + FROM public.context_worker_classify_file_import_internal( + requested_organization_id, requested_job_id, + requested_service_principal_id, requested_source_ref, + requested_resource_ref, requested_canonical_text, + requested_content_hash, requested_compiler_version, + requested_config_version, requested_signing_key_version, + requested_nonce, requested_issued_at, requested_expires_at + ); + IF decision.classification = 'unchanged' THEN + IF NOT EXISTS ( + SELECT 1 FROM public.file_revision_snapshot AS snapshot + WHERE snapshot.organization_id = requested_organization_id + AND snapshot.resource_ref = requested_resource_ref + AND snapshot.revision_id = decision.active_revision_id + AND snapshot.compilation_digest = requested_compilation_digest + AND snapshot.compilation_document IS NOT DISTINCT FROM + requested_compilation_document + AND ( + SELECT count(*) FROM public.context_fragment AS fragment + WHERE fragment.organization_id = requested_organization_id + AND fragment.resource_ref = requested_resource_ref + AND fragment.revision_id = decision.active_revision_id + ) = jsonb_array_length(requested_artifact_document) + AND NOT EXISTS ( + SELECT 1 FROM jsonb_array_elements( + requested_artifact_document + ) WITH ORDINALITY AS expected(fragment, ordinal) + WHERE NOT EXISTS ( + SELECT 1 FROM public.context_fragment AS fragment + WHERE fragment.organization_id = requested_organization_id + AND fragment.resource_ref = requested_resource_ref + AND fragment.revision_id = decision.active_revision_id + AND fragment.fragment_ref = expected.fragment->>'fragmentRef' + AND fragment.ordinal = (expected.ordinal - 1)::integer + AND fragment.content = expected.fragment->>'contextualText' + AND fragment.projection_kind = 'body' + ) + ) + AND NOT EXISTS ( + SELECT 1 + FROM jsonb_array_elements(requested_artifact_document) + AS expected(fragment) + CROSS JOIN LATERAL jsonb_array_elements_text( + expected.fragment->'searchPhrases' + ) AS phrase(value) + WHERE NOT EXISTS ( + SELECT 1 FROM public.exact_phrase_candidate AS candidate + WHERE candidate.organization_id = requested_organization_id + AND candidate.source_ref = requested_source_ref + AND candidate.resource_ref = requested_resource_ref + AND candidate.revision_id = decision.active_revision_id + AND candidate.fragment_ref = expected.fragment->>'fragmentRef' + AND candidate.phrase_digest = encode(public.digest( + convert_to('context-engine.exact-phrase.v1', 'UTF8') + || decode('00', 'hex') + || convert_to(phrase.value, 'UTF8'), 'sha256' + ), 'hex') + ) + ) + AND ( + SELECT count(*) + FROM public.exact_phrase_candidate AS candidate + WHERE candidate.organization_id = + requested_organization_id + AND candidate.resource_ref = requested_resource_ref + AND candidate.revision_id = + decision.active_revision_id + ) = ( + SELECT count(*) + FROM jsonb_array_elements(requested_artifact_document) + AS expected(fragment) + CROSS JOIN LATERAL jsonb_array_elements_text( + expected.fragment->'searchPhrases' + ) AS phrase(value) + ) + ) THEN + RAISE EXCEPTION 'no-op payload is not the active artifact' + USING ERRCODE = '22023'; + END IF; + SELECT COALESCE(max(event.ordinal), -1) + 1 INTO next_ordinal + FROM public.file_import_job_event AS event + WHERE event.organization_id = requested_organization_id + AND event.job_id = requested_job_id; + INSERT INTO public.file_import_job_event VALUES ( + requested_organization_id, requested_job_id, next_ordinal, + 'unchanged', 'active', job_row.lease_generation, + 'completed', decision.active_revision_id, + decision.reason_digest, now_at + ); + RETURN QUERY SELECT 'active'::text, 'initial'::text, + decision.active_revision_id, NULL::uuid, + decision.fragment_refs, decision.content_identity_digest, + 0::smallint, 'unchanged'::text, + decision.active_revision_id, decision.reason_digest; + RETURN; + ELSIF decision.classification IS NULL THEN + RETURN; + ELSIF decision.classification NOT IN ('publish', 'changed') THEN + RETURN; + END IF; + IF decision.classification = 'changed' THEN + SELECT resource.active_revision_id INTO old_revision + FROM public.context_resource AS resource + WHERE resource.organization_id = requested_organization_id + AND resource.resource_ref = requested_resource_ref + AND resource.source_ref = requested_source_ref + AND resource.active_revision_id IS NOT NULL + AND resource.tombstoned IS FALSE; + IF old_revision IS NULL THEN RETURN; END IF; + END IF; + IF NOT EXISTS ( + SELECT 1 + FROM public.file_acquisition AS acquisition + JOIN public.membership AS audience_membership + ON audience_membership.organization_id = acquisition.organization_id + AND audience_membership.membership_id = acquisition.audience_membership_id + AND audience_membership.membership_version = acquisition.audience_membership_version + AND audience_membership.status = 'active' + AND audience_membership.valid_from <= now_at + AND (audience_membership.valid_until IS NULL OR audience_membership.valid_until > now_at) + WHERE acquisition.organization_id = requested_organization_id + AND acquisition.acquisition_id = job_row.acquisition_id + AND acquisition.source_id = job_row.source_id + AND ( + decision.classification = 'publish' + OR ( + EXISTS ( + SELECT 1 FROM public.resource_access_policy AS access_policy + WHERE access_policy.organization_id = acquisition.organization_id + AND access_policy.resource_ref = requested_resource_ref + AND access_policy.principal_ref = acquisition.audience_principal_ref + AND access_policy.access_state = 'allowed' + ) + AND EXISTS ( + SELECT 1 FROM public.membership_resource_field_right AS field_right + WHERE field_right.organization_id = acquisition.organization_id + AND field_right.membership_id = acquisition.audience_membership_id + AND field_right.membership_version = acquisition.audience_membership_version + AND field_right.resource_ref = requested_resource_ref + AND field_right.field_ref = 'body' + ) + ) + ) + ) THEN RETURN; END IF; + INSERT INTO public.file_publication_recovery ( + organization_id, job_id, source_id, resource_ref, revision_id, + previous_revision_id, publication_kind, checkpoint, + content_identity_digest, content_hash, compilation_digest, + publication_payload_digest, compiler_version, config_version, + created_at, updated_at + ) VALUES ( + requested_organization_id, requested_job_id, job_row.source_id, + requested_resource_ref, requested_revision_id, old_revision, + CASE decision.classification + WHEN 'publish' THEN 'initial' ELSE 'replacement' + END, + 'acquired', decision.content_identity_digest, + requested_content_hash, requested_compilation_digest, + ({payload_digest}), + requested_compiler_version, requested_config_version, + now_at, now_at + ); + UPDATE public.file_import_job + SET resource_ref = requested_resource_ref, + revision_id = requested_revision_id + WHERE organization_id = requested_organization_id + AND job_id = requested_job_id AND state = 'running'; + IF NOT FOUND THEN RETURN; END IF; + SELECT COALESCE(max(event.ordinal), -1) + 1 INTO next_ordinal + FROM public.file_import_job_event AS event + WHERE event.organization_id = requested_organization_id + AND event.job_id = requested_job_id; + INSERT INTO public.file_import_job_event VALUES ( + requested_organization_id, requested_job_id, next_ordinal, + 'acquired', 'acquired', job_row.lease_generation, 'running', + requested_revision_id, NULL, now_at + ); + RETURN QUERY SELECT 'acquired'::text, + CASE decision.classification + WHEN 'publish' THEN 'initial' ELSE 'replacement' + END, + requested_revision_id, old_revision, NULL::text[], + decision.content_identity_digest, NULL::smallint, NULL::text, + NULL::uuid, NULL::text; + END; $function$ + """ + ) + + +def _create_prepare_function() -> None: + artifact_validation = _artifact_validation() + payload_digest = _publication_payload_digest() + op.execute( + f""" + CREATE FUNCTION public.context_worker_prepare_file_publication( + requested_organization_id uuid, requested_job_id uuid, + requested_service_principal_id uuid, requested_source_ref text, + requested_resource_ref text, requested_revision_id uuid, + requested_canonical_text text, + requested_compilation_document jsonb, + requested_artifact_document jsonb, + requested_lease_generation bigint, + requested_signing_key_version bigint, requested_nonce bytea, + requested_issued_at timestamptz, requested_expires_at timestamptz + ) RETURNS TABLE (checkpoint text, fragment_refs text[]) + LANGUAGE plpgsql SECURITY DEFINER + SET search_path = pg_catalog, pg_temp SET row_security = on + AS $function$ + DECLARE + job_row public.file_import_job%ROWTYPE; + recovery_row public.file_publication_recovery%ROWTYPE; + first_fragment text; + prepared_fragments text[]; + now_at timestamptz; + next_ordinal bigint; + BEGIN + IF SESSION_USER <> '{_WORKER}' + OR NOT ({artifact_validation}) + THEN RETURN; END IF; + PERFORM pg_catalog.set_config('app.organization_id', requested_organization_id::text, true); + PERFORM pg_catalog.set_config('app.worker_job_id', requested_job_id::text, true); + now_at := pg_catalog.statement_timestamp(); + SELECT * INTO job_row FROM public.file_import_job AS job + WHERE job.organization_id = requested_organization_id + AND job.job_id = requested_job_id + AND job.service_principal_id = requested_service_principal_id + AND job.source_id::text = requested_source_ref + AND job.state = 'running' + AND job.resource_ref = requested_resource_ref + AND job.revision_id = requested_revision_id + AND job.lease_generation = requested_lease_generation + AND job.signing_key_version = requested_signing_key_version + AND job.lease_nonce_digest = public.digest(requested_nonce, 'sha256') + AND job.lease_issued_at = requested_issued_at + AND job.lease_expires_at = requested_expires_at + AND now_at < job.lease_expires_at + AND EXISTS (SELECT 1 FROM public.service_principal AS principal + WHERE principal.organization_id = job.organization_id + AND principal.service_principal_id = job.service_principal_id + AND principal.workload = job.workload + AND principal.worker_audience = job.worker_audience + AND principal.operation = job.operation + AND principal.enabled IS TRUE) + FOR UPDATE; + IF job_row.job_id IS NULL THEN RETURN; END IF; + SELECT * INTO recovery_row + FROM public.file_publication_recovery AS recovery + WHERE recovery.organization_id = requested_organization_id + AND recovery.job_id = requested_job_id + AND recovery.resource_ref = requested_resource_ref + AND recovery.revision_id = requested_revision_id + AND recovery.checkpoint = 'acquired' + AND encode(public.digest( + convert_to(requested_canonical_text, 'UTF8'), 'sha256' + ), 'hex') = recovery.content_hash + AND ( + (recovery.compiler_version = 'context-engine-markdown-v1' + AND requested_compilation_document IS NULL) + OR + (recovery.compiler_version = 'context-engine-markdown-v2' + AND requested_compilation_document->>'compilationDigest' + = recovery.compilation_digest) + ) + AND recovery.publication_payload_digest = ({payload_digest}) + FOR UPDATE; + IF recovery_row.job_id IS NULL THEN RETURN; END IF; + PERFORM 1 FROM public.file_resource_ingestion_guard AS guard + WHERE guard.organization_id = requested_organization_id + AND guard.source_id = job_row.source_id + AND guard.resource_ref = requested_resource_ref + FOR UPDATE; + IF NOT FOUND THEN RETURN; END IF; + IF NOT EXISTS ( + SELECT 1 FROM public.file_acquisition AS acquisition + JOIN public.membership AS membership + ON membership.organization_id = acquisition.organization_id + AND membership.membership_id = acquisition.audience_membership_id + AND membership.membership_version = acquisition.audience_membership_version + AND membership.status = 'active' + AND membership.valid_from <= now_at + AND (membership.valid_until IS NULL OR membership.valid_until > now_at) + WHERE acquisition.organization_id = requested_organization_id + AND acquisition.acquisition_id = job_row.acquisition_id + AND acquisition.source_id = job_row.source_id + AND ( + recovery_row.publication_kind = 'initial' + OR ( + EXISTS ( + SELECT 1 FROM public.resource_access_policy AS access_policy + WHERE access_policy.organization_id = acquisition.organization_id + AND access_policy.resource_ref = requested_resource_ref + AND access_policy.principal_ref = acquisition.audience_principal_ref + AND access_policy.access_state = 'allowed' + ) + AND EXISTS ( + SELECT 1 FROM public.membership_resource_field_right AS field_right + WHERE field_right.organization_id = acquisition.organization_id + AND field_right.membership_id = acquisition.audience_membership_id + AND field_right.membership_version = acquisition.audience_membership_version + AND field_right.resource_ref = requested_resource_ref + AND field_right.field_ref = 'body' + ) + ) + ) + ) THEN RETURN; END IF; + SET CONSTRAINTS ALL DEFERRED; + IF recovery_row.publication_kind = 'initial' THEN + INSERT INTO public.context_resource ( + organization_id, resource_ref, source_ref, + active_revision_id, tombstoned + ) VALUES ( + requested_organization_id, requested_resource_ref, + requested_source_ref, NULL, false + ) ON CONFLICT (organization_id, resource_ref) DO NOTHING; + IF NOT EXISTS ( + SELECT 1 FROM public.context_resource AS resource + WHERE resource.organization_id = requested_organization_id + AND resource.resource_ref = requested_resource_ref + AND resource.source_ref = requested_source_ref + AND resource.active_revision_id IS NULL + AND resource.tombstoned IS FALSE + ) THEN RETURN; END IF; + ELSIF NOT EXISTS ( + SELECT 1 FROM public.context_resource AS resource + WHERE resource.organization_id = requested_organization_id + AND resource.resource_ref = requested_resource_ref + AND resource.source_ref = requested_source_ref + AND resource.active_revision_id = recovery_row.previous_revision_id + AND resource.tombstoned IS FALSE + ) THEN RETURN; + END IF; + INSERT INTO public.context_revision VALUES ( + requested_organization_id, requested_resource_ref, + requested_revision_id + ); + INSERT INTO public.file_revision_snapshot ( + organization_id, resource_ref, revision_id, acquisition_id, + canonical_text, content_hash, compilation_digest, + compiler_version, config_version, compilation_document + ) VALUES ( + requested_organization_id, requested_resource_ref, + requested_revision_id, job_row.acquisition_id, + requested_canonical_text, + recovery_row.content_hash, recovery_row.compilation_digest, + recovery_row.compiler_version, recovery_row.config_version, + requested_compilation_document + ); + INSERT INTO public.context_fragment ( + organization_id, resource_ref, revision_id, fragment_ref, + ordinal, content, projection_kind + ) + SELECT requested_organization_id, requested_resource_ref, + requested_revision_id, item.fragment->>'fragmentRef', + (item.ordinal - 1)::integer, + item.fragment->>'contextualText', 'body' + FROM jsonb_array_elements(requested_artifact_document) + WITH ORDINALITY AS item(fragment, ordinal) + ORDER BY item.ordinal; + INSERT INTO public.revision_publication_event VALUES ( + requested_organization_id, requested_resource_ref, + requested_revision_id, 0, 'prepared', now_at + ); + IF recovery_row.publication_kind = 'initial' THEN + INSERT INTO public.resource_access_policy + SELECT requested_organization_id, requested_resource_ref, + acquisition.audience_principal_ref, 1, 'allowed', NULL + FROM public.file_acquisition AS acquisition + WHERE acquisition.organization_id = requested_organization_id + AND acquisition.acquisition_id = job_row.acquisition_id; + INSERT INTO public.membership_resource_field_right + SELECT requested_organization_id, + acquisition.audience_membership_id, + acquisition.audience_membership_version, + requested_resource_ref, 'body' + FROM public.file_acquisition AS acquisition + WHERE acquisition.organization_id = requested_organization_id + AND acquisition.acquisition_id = job_row.acquisition_id; + END IF; + SELECT array_agg(fragment_ref ORDER BY ordinal), + (array_agg(fragment_ref ORDER BY ordinal))[1] + INTO prepared_fragments, first_fragment + FROM public.context_fragment + WHERE organization_id = requested_organization_id + AND resource_ref = requested_resource_ref + AND revision_id = requested_revision_id; + UPDATE public.file_publication_recovery AS recovery + SET checkpoint = 'prepared', updated_at = now_at + WHERE recovery.organization_id = requested_organization_id + AND recovery.job_id = requested_job_id + AND recovery.checkpoint = 'acquired'; + IF NOT FOUND THEN RETURN; END IF; + UPDATE public.file_import_job + SET state = 'prepared', fragment_ref = first_fragment + WHERE organization_id = requested_organization_id + AND job_id = requested_job_id AND state = 'running'; + IF NOT FOUND THEN RETURN; END IF; + SELECT COALESCE(max(event.ordinal), -1) + 1 INTO next_ordinal + FROM public.file_import_job_event AS event + WHERE event.organization_id = requested_organization_id + AND event.job_id = requested_job_id; + INSERT INTO public.file_import_job_event VALUES ( + requested_organization_id, requested_job_id, next_ordinal, + 'prepared', 'prepared', job_row.lease_generation, 'prepared', + requested_revision_id, NULL, now_at + ); + RETURN QUERY SELECT 'prepared'::text, prepared_fragments; + END; $function$ + """ + ) + + +def _create_index_function() -> None: + artifact_validation = _artifact_validation() + payload_digest = _publication_payload_digest() + op.execute( + f""" + CREATE FUNCTION public.context_worker_index_file_publication( + requested_organization_id uuid, requested_job_id uuid, + requested_service_principal_id uuid, requested_source_ref text, + requested_resource_ref text, requested_revision_id uuid, + requested_canonical_text text, + requested_compilation_document jsonb, + requested_artifact_document jsonb, + requested_lease_generation bigint, + requested_signing_key_version bigint, requested_nonce bytea, + requested_issued_at timestamptz, requested_expires_at timestamptz + ) RETURNS TABLE (checkpoint text, fragment_refs text[]) + LANGUAGE plpgsql SECURITY DEFINER + SET search_path = pg_catalog, pg_temp SET row_security = on + AS $function$ + DECLARE + job_row public.file_import_job%ROWTYPE; + recovery_row public.file_publication_recovery%ROWTYPE; + indexed_fragments text[]; + now_at timestamptz; + next_ordinal bigint; + BEGIN + IF SESSION_USER <> '{_WORKER}' OR NOT ({artifact_validation}) + THEN RETURN; END IF; + PERFORM pg_catalog.set_config('app.organization_id', requested_organization_id::text, true); + PERFORM pg_catalog.set_config('app.worker_job_id', requested_job_id::text, true); + now_at := pg_catalog.statement_timestamp(); + SELECT * INTO job_row FROM public.file_import_job AS job + WHERE job.organization_id = requested_organization_id + AND job.job_id = requested_job_id + AND job.service_principal_id = requested_service_principal_id + AND job.source_id::text = requested_source_ref + AND job.state = 'prepared' + AND job.resource_ref = requested_resource_ref + AND job.revision_id = requested_revision_id + AND job.lease_generation = requested_lease_generation + AND job.signing_key_version = requested_signing_key_version + AND job.lease_nonce_digest = public.digest(requested_nonce, 'sha256') + AND job.lease_issued_at = requested_issued_at + AND job.lease_expires_at = requested_expires_at + AND now_at < job.lease_expires_at + AND EXISTS (SELECT 1 FROM public.service_principal AS principal + WHERE principal.organization_id = job.organization_id + AND principal.service_principal_id = job.service_principal_id + AND principal.workload = job.workload + AND principal.worker_audience = job.worker_audience + AND principal.operation = job.operation + AND principal.enabled IS TRUE) + FOR UPDATE; + IF job_row.job_id IS NULL THEN RETURN; END IF; + SELECT * INTO recovery_row + FROM public.file_publication_recovery AS recovery + WHERE recovery.organization_id = requested_organization_id + AND recovery.job_id = requested_job_id + AND recovery.resource_ref = requested_resource_ref + AND recovery.revision_id = requested_revision_id + AND recovery.checkpoint = 'prepared' + AND encode(public.digest( + convert_to(requested_canonical_text, 'UTF8'), 'sha256' + ), 'hex') = recovery.content_hash + AND recovery.publication_payload_digest = ({payload_digest}) + FOR UPDATE; + IF recovery_row.job_id IS NULL + OR NOT EXISTS ( + SELECT 1 FROM public.file_revision_snapshot AS snapshot + WHERE snapshot.organization_id = requested_organization_id + AND snapshot.resource_ref = requested_resource_ref + AND snapshot.revision_id = requested_revision_id + AND snapshot.content_hash = recovery_row.content_hash + AND snapshot.canonical_text = requested_canonical_text + AND snapshot.compilation_digest = recovery_row.compilation_digest + AND snapshot.compilation_document IS NOT DISTINCT FROM requested_compilation_document + AND ( + SELECT count(*) FROM public.context_fragment AS fragment + WHERE fragment.organization_id = requested_organization_id + AND fragment.resource_ref = requested_resource_ref + AND fragment.revision_id = requested_revision_id + ) = jsonb_array_length(requested_artifact_document) + ) + THEN RETURN; END IF; + INSERT INTO public.exact_phrase_candidate ( + organization_id, phrase_digest, source_ref, resource_ref, + revision_id, fragment_ref + ) + SELECT requested_organization_id, + encode(public.digest( + convert_to('context-engine.exact-phrase.v1', 'UTF8') + || decode('00', 'hex') + || convert_to(phrase.value, 'UTF8'), 'sha256' + ), 'hex'), requested_source_ref, requested_resource_ref, + requested_revision_id, item.fragment->>'fragmentRef' + FROM jsonb_array_elements(requested_artifact_document) + WITH ORDINALITY AS item(fragment, fragment_ordinal) + CROSS JOIN LATERAL jsonb_array_elements_text( + item.fragment->'searchPhrases' + ) WITH ORDINALITY AS phrase(value, phrase_ordinal) + ORDER BY item.fragment_ordinal, phrase.phrase_ordinal; + INSERT INTO public.revision_publication_event VALUES ( + requested_organization_id, requested_resource_ref, + requested_revision_id, 1, 'indexed', now_at + ); + SELECT array_agg(fragment_ref ORDER BY ordinal) + INTO indexed_fragments + FROM public.context_fragment + WHERE organization_id = requested_organization_id + AND resource_ref = requested_resource_ref + AND revision_id = requested_revision_id; + IF recovery_row.publication_kind = 'replacement' THEN + INSERT INTO public.file_revision_replacement_plan ( + organization_id, source_id, resource_ref, + previous_revision_id, replacement_revision_id, + acquisition_id, job_id, content_identity_digest, prepared_at + ) VALUES ( + requested_organization_id, job_row.source_id, + requested_resource_ref, recovery_row.previous_revision_id, + requested_revision_id, job_row.acquisition_id, + requested_job_id, recovery_row.content_identity_digest, + now_at + ); + END IF; + UPDATE public.file_publication_recovery AS recovery + SET checkpoint = 'ready', updated_at = now_at + WHERE recovery.organization_id = requested_organization_id + AND recovery.job_id = requested_job_id + AND recovery.checkpoint = 'prepared'; + IF NOT FOUND THEN RETURN; END IF; + UPDATE public.file_import_job + SET state = 'ready' + WHERE organization_id = requested_organization_id + AND job_id = requested_job_id AND state = 'prepared'; + IF NOT FOUND THEN RETURN; END IF; + SELECT COALESCE(max(event.ordinal), -1) + 1 INTO next_ordinal + FROM public.file_import_job_event AS event + WHERE event.organization_id = requested_organization_id + AND event.job_id = requested_job_id; + INSERT INTO public.file_import_job_event VALUES ( + requested_organization_id, requested_job_id, next_ordinal, + 'indexed', 'indexed', job_row.lease_generation, 'ready', + requested_revision_id, NULL, now_at + ); + RETURN QUERY SELECT 'ready'::text, indexed_fragments; + END; $function$ + """ + ) + + +def _create_activate_function() -> None: + op.execute( + f""" + CREATE FUNCTION public.context_worker_activate_recoverable_file_publication( + requested_organization_id uuid, requested_job_id uuid, + requested_service_principal_id uuid, requested_source_ref text, + requested_resource_ref text, requested_revision_id uuid, + requested_lease_generation bigint, + requested_signing_key_version bigint, requested_nonce bytea, + requested_issued_at timestamptz, requested_expires_at timestamptz + ) RETURNS TABLE ( + effect_count smallint, outcome text, active_revision_id uuid, + fragment_refs text[], content_identity_digest text, + reason_digest text + ) + LANGUAGE plpgsql SECURITY DEFINER + SET search_path = pg_catalog, pg_temp SET row_security = on + AS $function$ + DECLARE + job_row public.file_import_job%ROWTYPE; + recovery_row public.file_publication_recovery%ROWTYPE; + activated record; + active_fragments text[]; + now_at timestamptz; + next_ordinal bigint; + BEGIN + IF SESSION_USER <> '{_WORKER}' THEN RETURN; END IF; + PERFORM pg_catalog.set_config('app.organization_id', requested_organization_id::text, true); + PERFORM pg_catalog.set_config('app.worker_job_id', requested_job_id::text, true); + now_at := pg_catalog.statement_timestamp(); + SELECT * INTO job_row FROM public.file_import_job AS job + WHERE job.organization_id = requested_organization_id + AND job.job_id = requested_job_id + AND job.service_principal_id = requested_service_principal_id + AND job.source_id::text = requested_source_ref + AND job.state = 'ready' + AND job.resource_ref = requested_resource_ref + AND job.revision_id = requested_revision_id + AND job.lease_generation = requested_lease_generation + AND job.signing_key_version = requested_signing_key_version + AND job.lease_nonce_digest = public.digest(requested_nonce, 'sha256') + AND job.lease_issued_at = requested_issued_at + AND job.lease_expires_at = requested_expires_at + AND now_at < job.lease_expires_at + AND EXISTS ( + SELECT 1 FROM public.service_principal AS principal + WHERE principal.organization_id = job.organization_id + AND principal.service_principal_id = job.service_principal_id + AND principal.workload = job.workload + AND principal.worker_audience = job.worker_audience + AND principal.operation = job.operation + AND principal.enabled IS TRUE + ) + FOR UPDATE; + IF job_row.job_id IS NULL THEN RETURN; END IF; + SELECT * INTO recovery_row + FROM public.file_publication_recovery AS recovery + WHERE recovery.organization_id = requested_organization_id + AND recovery.job_id = requested_job_id + AND recovery.resource_ref = requested_resource_ref + AND recovery.revision_id = requested_revision_id + AND recovery.checkpoint = 'ready' + FOR UPDATE; + IF recovery_row.job_id IS NULL THEN RETURN; END IF; + IF recovery_row.publication_kind = 'replacement' THEN + SELECT * INTO activated + FROM public.context_worker_activate_file_replacement( + requested_organization_id, requested_job_id, + requested_service_principal_id, requested_source_ref, + requested_resource_ref, + recovery_row.previous_revision_id, + requested_revision_id, requested_signing_key_version, + requested_nonce, requested_issued_at, requested_expires_at + ); + IF activated.effect_count IS DISTINCT FROM 1 THEN RETURN; END IF; + effect_count := activated.effect_count; + outcome := activated.outcome; + active_revision_id := activated.active_revision_id; + fragment_refs := activated.fragment_refs; + content_identity_digest := activated.content_identity_digest; + reason_digest := activated.reason_digest; + ELSE + PERFORM pg_catalog.pg_advisory_xact_lock( + pg_catalog.hashtextextended( + 'context-engine.file-publication:' + || requested_organization_id::text, 0 + ) + ); + PERFORM 1 FROM public.file_resource_ingestion_guard AS guard + WHERE guard.organization_id = requested_organization_id + AND guard.source_id = job_row.source_id + AND guard.resource_ref = requested_resource_ref + FOR UPDATE; + IF NOT FOUND OR NOT EXISTS ( + SELECT 1 FROM public.context_resource AS resource + WHERE resource.organization_id = requested_organization_id + AND resource.resource_ref = requested_resource_ref + AND resource.source_ref = requested_source_ref + AND resource.active_revision_id IS NULL + AND resource.tombstoned IS FALSE + AND EXISTS ( + SELECT 1 + FROM public.file_acquisition AS acquisition + JOIN public.membership AS audience_membership + ON audience_membership.organization_id = acquisition.organization_id + AND audience_membership.membership_id = acquisition.audience_membership_id + AND audience_membership.membership_version = acquisition.audience_membership_version + AND audience_membership.status = 'active' + AND audience_membership.valid_from <= now_at + AND (audience_membership.valid_until IS NULL OR audience_membership.valid_until > now_at) + JOIN public.resource_access_policy AS access_policy + ON access_policy.organization_id = acquisition.organization_id + AND access_policy.resource_ref = requested_resource_ref + AND access_policy.principal_ref = acquisition.audience_principal_ref + AND access_policy.access_state = 'allowed' + JOIN public.membership_resource_field_right AS field_right + ON field_right.organization_id = acquisition.organization_id + AND field_right.membership_id = acquisition.audience_membership_id + AND field_right.membership_version = acquisition.audience_membership_version + AND field_right.resource_ref = requested_resource_ref + AND field_right.field_ref = 'body' + WHERE acquisition.organization_id = requested_organization_id + AND acquisition.acquisition_id = job_row.acquisition_id + AND acquisition.source_id = job_row.source_id + ) + AND ( + SELECT array_agg(event.state ORDER BY event.ordinal) + FROM public.revision_publication_event AS event + WHERE event.organization_id = requested_organization_id + AND event.resource_ref = requested_resource_ref + AND event.revision_id = requested_revision_id + ) = ARRAY['prepared', 'indexed']::text[] + AND NOT EXISTS ( + SELECT 1 FROM public.context_fragment AS fragment + WHERE fragment.organization_id = requested_organization_id + AND fragment.resource_ref = requested_resource_ref + AND fragment.revision_id = requested_revision_id + AND NOT EXISTS ( + SELECT 1 FROM public.exact_phrase_candidate AS candidate + WHERE candidate.organization_id = fragment.organization_id + AND candidate.resource_ref = fragment.resource_ref + AND candidate.revision_id = fragment.revision_id + AND candidate.fragment_ref = fragment.fragment_ref + ) + ) + ) THEN RETURN; END IF; + SELECT array_agg(fragment_ref ORDER BY ordinal) + INTO active_fragments FROM public.context_fragment + WHERE organization_id = requested_organization_id + AND resource_ref = requested_resource_ref + AND revision_id = requested_revision_id; + UPDATE public.context_resource AS resource + SET active_revision_id = requested_revision_id + WHERE resource.organization_id = requested_organization_id + AND resource.resource_ref = requested_resource_ref + AND resource.active_revision_id IS NULL; + IF NOT FOUND THEN RETURN; END IF; + INSERT INTO public.revision_publication_event VALUES ( + requested_organization_id, requested_resource_ref, + requested_revision_id, 2, 'active', now_at + ); + UPDATE public.file_import_job + SET state = 'completed', completed_at = now_at, effect_count = 1 + WHERE organization_id = requested_organization_id + AND job_id = requested_job_id AND state = 'ready'; + IF NOT FOUND THEN RETURN; END IF; + effect_count := 1; + outcome := 'published'; + active_revision_id := requested_revision_id; + fragment_refs := active_fragments; + content_identity_digest := recovery_row.content_identity_digest; + reason_digest := NULL; + END IF; + UPDATE public.file_publication_recovery AS recovery + SET checkpoint = 'completed', updated_at = now_at + WHERE recovery.organization_id = requested_organization_id + AND recovery.job_id = requested_job_id + AND recovery.checkpoint = 'ready'; + IF NOT FOUND THEN RETURN; END IF; + SELECT COALESCE(max(event.ordinal), -1) + 1 INTO next_ordinal + FROM public.file_import_job_event AS event + WHERE event.organization_id = requested_organization_id + AND event.job_id = requested_job_id; + INSERT INTO public.file_import_job_event VALUES ( + requested_organization_id, requested_job_id, next_ordinal, + 'active', 'active', job_row.lease_generation, 'completed', + requested_revision_id, NULL, now_at + ); + RETURN NEXT; + END; $function$ + """ + ) + + +def _create_interrupt_function() -> None: + op.execute( + f""" + CREATE FUNCTION public.context_worker_record_file_import_interruption( + requested_organization_id uuid, requested_job_id uuid, + requested_service_principal_id uuid, requested_source_ref text, + requested_boundary text, requested_lease_generation bigint, + requested_signing_key_version bigint, + requested_nonce bytea, requested_issued_at timestamptz, + requested_expires_at timestamptz + ) RETURNS boolean + LANGUAGE plpgsql SECURITY DEFINER + SET search_path = pg_catalog, pg_temp SET row_security = on + AS $function$ + DECLARE + job_row public.file_import_job%ROWTYPE; + expected_state text; + expected_checkpoint text; + next_ordinal bigint; + now_at timestamptz; + BEGIN + IF SESSION_USER <> '{_WORKER}' + OR requested_boundary NOT IN ('acquired', 'prepared', 'indexed') + THEN RETURN false; END IF; + expected_state := CASE requested_boundary + WHEN 'acquired' THEN 'running' + WHEN 'prepared' THEN 'prepared' + ELSE 'ready' + END; + expected_checkpoint := CASE requested_boundary + WHEN 'indexed' THEN 'ready' ELSE requested_boundary + END; + PERFORM pg_catalog.set_config('app.organization_id', requested_organization_id::text, true); + PERFORM pg_catalog.set_config('app.worker_job_id', requested_job_id::text, true); + now_at := pg_catalog.statement_timestamp(); + SELECT * INTO job_row FROM public.file_import_job AS job + WHERE job.organization_id = requested_organization_id + AND job.job_id = requested_job_id + AND job.service_principal_id = requested_service_principal_id + AND job.source_id::text = requested_source_ref + AND job.state = expected_state + AND job.lease_generation = requested_lease_generation + AND job.signing_key_version = requested_signing_key_version + AND job.lease_nonce_digest = public.digest(requested_nonce, 'sha256') + AND job.lease_issued_at = requested_issued_at + AND job.lease_expires_at = requested_expires_at + AND now_at < job.lease_expires_at + AND EXISTS ( + SELECT 1 FROM public.file_publication_recovery AS recovery + WHERE recovery.organization_id = job.organization_id + AND recovery.job_id = job.job_id + AND recovery.checkpoint = expected_checkpoint + ) + FOR UPDATE; + IF job_row.job_id IS NULL THEN RETURN false; END IF; + SELECT COALESCE(max(event.ordinal), -1) + 1 INTO next_ordinal + FROM public.file_import_job_event AS event + WHERE event.organization_id = requested_organization_id + AND event.job_id = requested_job_id; + INSERT INTO public.file_import_job_event VALUES ( + requested_organization_id, requested_job_id, next_ordinal, + 'interrupted', requested_boundary, job_row.lease_generation, + job_row.state, job_row.revision_id, + encode(public.digest( + convert_to('context-engine.file-interruption.v1', 'UTF8') + || decode('00', 'hex') + || uuid_send(requested_organization_id) + || uuid_send(requested_job_id) + || int8send(job_row.lease_generation) + || convert_to(requested_boundary, 'UTF8'), 'sha256' + ), 'hex'), now_at + ); + RETURN true; + END; $function$ + """ + ) + + +def _restore_non_recoverable_lease_functions() -> None: + """Restore the Issue #23 lease behavior before removing recovery columns.""" + + op.execute( + f""" + CREATE OR REPLACE FUNCTION public.context_worker_issue_file_import_lease( + requested_organization_id uuid, requested_job_id uuid, + requested_service_principal_id uuid, requested_source_ref text, + requested_signing_key_version bigint, requested_nonce bytea, + requested_ttl_seconds integer + ) RETURNS TABLE (issued_at timestamptz, expires_at timestamptz) + LANGUAGE plpgsql SECURITY DEFINER + SET search_path = pg_catalog, pg_temp SET row_security = on + AS $function$ + DECLARE now_at timestamptz; + BEGIN + IF SESSION_USER <> '{_CONTROL}' + OR requested_signing_key_version NOT BETWEEN 1 AND {_MAX_BIGINT} + OR pg_catalog.octet_length(requested_nonce) <> 32 + OR requested_ttl_seconds NOT BETWEEN 1 AND {_MAX_TTL} + THEN RETURN; END IF; + PERFORM pg_catalog.set_config( + 'app.organization_id', requested_organization_id::text, true + ); + PERFORM pg_catalog.set_config( + 'app.worker_job_id', requested_job_id::text, true + ); + now_at := pg_catalog.date_trunc( + 'second', pg_catalog.transaction_timestamp() + ); + UPDATE public.file_import_job AS job + SET state = 'leased', + signing_key_version = requested_signing_key_version, + lease_nonce_digest = public.digest(requested_nonce, 'sha256'), + lease_issued_at = now_at, + lease_expires_at = now_at + pg_catalog.make_interval( + secs => requested_ttl_seconds + ) + WHERE job.organization_id = requested_organization_id + AND job.job_id = requested_job_id + AND job.service_principal_id = requested_service_principal_id + AND job.source_id::text = requested_source_ref + AND job.state = 'available' + AND EXISTS ( + SELECT 1 FROM public.service_principal AS principal + WHERE principal.organization_id = job.organization_id + AND principal.service_principal_id = job.service_principal_id + AND principal.workload = job.workload + AND principal.worker_audience = job.worker_audience + AND principal.operation = job.operation + AND principal.enabled IS TRUE + ) + RETURNING job.lease_issued_at, job.lease_expires_at + INTO issued_at, expires_at; + IF issued_at IS NOT NULL THEN RETURN NEXT; END IF; + END; $function$ + """ + ) + op.execute( + f""" + CREATE OR REPLACE FUNCTION public.context_worker_redeem_file_import( + requested_organization_id uuid, requested_job_id uuid, + requested_service_principal_id uuid, requested_source_ref text, + requested_signing_key_version bigint, requested_nonce bytea, + requested_issued_at timestamptz, + requested_expires_at timestamptz + ) RETURNS TABLE ( + source_ref text, root_ref text, relative_path text, + audience_principal_ref text, audience_membership_id uuid, + audience_membership_version bigint, acquisition_id uuid + ) + LANGUAGE plpgsql SECURITY DEFINER + SET search_path = pg_catalog, pg_temp SET row_security = on + AS $function$ + DECLARE redeemed_at timestamptz; + BEGIN + IF SESSION_USER <> '{_WORKER}' THEN RETURN; END IF; + PERFORM pg_catalog.set_config( + 'app.organization_id', requested_organization_id::text, true + ); + PERFORM pg_catalog.set_config( + 'app.worker_job_id', requested_job_id::text, true + ); + redeemed_at := pg_catalog.statement_timestamp(); + UPDATE public.file_import_job AS job + SET state = 'running', lease_redeemed_at = redeemed_at + WHERE job.organization_id = requested_organization_id + AND job.job_id = requested_job_id + AND job.service_principal_id = requested_service_principal_id + AND job.source_id::text = requested_source_ref + AND job.state = 'leased' + AND job.signing_key_version = requested_signing_key_version + AND job.lease_nonce_digest = public.digest(requested_nonce, 'sha256') + AND job.lease_issued_at = requested_issued_at + AND job.lease_expires_at = requested_expires_at + AND redeemed_at >= job.lease_issued_at + AND redeemed_at < job.lease_expires_at + AND EXISTS ( + SELECT 1 FROM public.service_principal AS principal + WHERE principal.organization_id = job.organization_id + AND principal.service_principal_id = job.service_principal_id + AND principal.workload = job.workload + AND principal.worker_audience = job.worker_audience + AND principal.operation = job.operation + AND principal.enabled IS TRUE + ); + IF NOT FOUND THEN RETURN; END IF; + RETURN QUERY + SELECT job.source_id::text, version.root_ref, + acquisition.relative_path, + acquisition.audience_principal_ref, + acquisition.audience_membership_id, + acquisition.audience_membership_version, + acquisition.acquisition_id + FROM public.file_import_job AS job + JOIN public.file_acquisition AS acquisition + ON acquisition.organization_id = job.organization_id + AND acquisition.acquisition_id = job.acquisition_id + JOIN public.source_version AS version + ON version.organization_id = acquisition.organization_id + AND version.source_id = acquisition.source_id + AND version.version_id = acquisition.source_version_id + WHERE job.organization_id = requested_organization_id + AND job.job_id = requested_job_id; + END; $function$ + """ + ) + + +def downgrade() -> None: + """Remove recovery only when no resumable publication would be lost.""" + + op.execute( + "LOCK TABLE file_publication_recovery, file_import_job_event " + "IN ACCESS EXCLUSIVE MODE" + ) + bind = op.get_bind() + if bind.execute( + sa.text( + "SELECT EXISTS (" + "SELECT 1 FROM file_publication_recovery " + "WHERE checkpoint <> 'completed' " + "UNION ALL " + "SELECT 1 FROM file_import_job_event AS event " + "JOIN file_import_job AS job " + "ON job.organization_id = event.organization_id " + "AND job.job_id = event.job_id " + "WHERE job.state <> 'completed'" + ")" + ) + ).scalar_one(): + raise RuntimeError( + "File recovery downgrade requires no resumable recovery rows" + ) + for name, signature in ( + ("context_worker_record_file_import_interruption", _INTERRUPT_SIGNATURE), + ("context_worker_activate_recoverable_file_publication", _ACTIVATE_SIGNATURE), + ("context_worker_index_file_publication", _STEP_SIGNATURE), + ("context_worker_prepare_file_publication", _STEP_SIGNATURE), + ("context_worker_acquire_file_publication", _ACQUIRE_SIGNATURE), + ("context_worker_fail_file_import", _FAIL_SIGNATURE), + ("context_worker_redeem_file_import", _REDEEM_SIGNATURE), + ("context_worker_issue_file_import_lease", _ISSUE_SIGNATURE), + ("context_worker_activate_file_replacement", _LEGACY_ACTIVATE_GENERATION_SIGNATURE), + ("context_worker_stage_structural_file_replacement", _STAGE_V2_GENERATION_SIGNATURE), + ("context_worker_stage_file_replacement", _STAGE_V1_GENERATION_SIGNATURE), + ("context_worker_publish_structural_file_import_v2", _PUBLISH_V2_GENERATION_SIGNATURE), + ("context_worker_publish_file_import_v2", _PUBLISH_V1_GENERATION_SIGNATURE), + ): + op.execute(f"DROP FUNCTION public.{name}{signature}") + op.execute(f"GRANT CREATE ON SCHEMA public TO {_DEFINER}") + op.execute(f"SET LOCAL ROLE {_DEFINER}") + _restore_non_recoverable_lease_functions() + _restore_non_recoverable_fail_function() + for name, signature in ( + ("context_worker_publish_file_import_v2", _PUBLISH_V1_SIGNATURE), + ("context_worker_publish_structural_file_import_v2", _PUBLISH_V2_SIGNATURE), + ("context_worker_stage_file_replacement", _STAGE_V1_SIGNATURE), + ("context_worker_stage_structural_file_replacement", _STAGE_V2_SIGNATURE), + ("context_worker_activate_file_replacement", _LEGACY_ACTIVATE_SIGNATURE), + ): + op.execute(f"GRANT EXECUTE ON FUNCTION public.{name}{signature} TO {_WORKER}") + op.execute("RESET ROLE") + op.execute(f"REVOKE CREATE ON SCHEMA public FROM {_DEFINER}") + op.drop_table("file_import_job_event") + op.drop_table("file_publication_recovery") + op.drop_constraint( + "ck_file_import_job_recovery_from_state", "file_import_job", type_="check" + ) + op.drop_constraint( + "ck_file_import_job_state_consistency", "file_import_job", type_="check" + ) + op.drop_constraint("ck_file_import_job_state", "file_import_job", type_="check") + op.drop_column("file_import_job", "recovery_from_state") + op.drop_column("file_import_job", "lease_generation") + op.create_check_constraint( + "ck_file_import_job_state", + "file_import_job", + "state IN ('available', 'leased', 'running', 'ready', 'failed', 'completed')", + ) + op.create_check_constraint( + "ck_file_import_job_state_consistency", + "file_import_job", + "(state = 'available' AND signing_key_version IS NULL AND lease_nonce_digest IS NULL AND lease_issued_at IS NULL AND lease_expires_at IS NULL AND lease_redeemed_at IS NULL AND failed_at IS NULL AND completed_at IS NULL AND resource_ref IS NULL AND revision_id IS NULL AND fragment_ref IS NULL AND effect_count = 0) OR " + "(state = 'leased' AND signing_key_version > 0 AND octet_length(lease_nonce_digest) = 32 AND lease_issued_at IS NOT NULL AND lease_expires_at > lease_issued_at AND lease_redeemed_at IS NULL AND failed_at IS NULL AND completed_at IS NULL AND resource_ref IS NULL AND revision_id IS NULL AND fragment_ref IS NULL AND effect_count = 0) OR " + "(state = 'running' AND signing_key_version > 0 AND octet_length(lease_nonce_digest) = 32 AND lease_issued_at IS NOT NULL AND lease_expires_at > lease_issued_at AND lease_redeemed_at >= lease_issued_at AND failed_at IS NULL AND completed_at IS NULL AND resource_ref IS NULL AND revision_id IS NULL AND fragment_ref IS NULL AND effect_count = 0) OR " + "(state = 'ready' AND signing_key_version > 0 AND octet_length(lease_nonce_digest) = 32 AND lease_issued_at IS NOT NULL AND lease_expires_at > lease_issued_at AND lease_redeemed_at >= lease_issued_at AND failed_at IS NULL AND completed_at IS NULL AND resource_ref IS NOT NULL AND revision_id IS NOT NULL AND fragment_ref IS NOT NULL AND effect_count = 0) OR " + "(state = 'failed' AND signing_key_version > 0 AND octet_length(lease_nonce_digest) = 32 AND lease_issued_at IS NOT NULL AND lease_expires_at > lease_issued_at AND lease_redeemed_at >= lease_issued_at AND failed_at >= lease_redeemed_at AND completed_at IS NULL AND resource_ref IS NULL AND revision_id IS NULL AND fragment_ref IS NULL AND effect_count = 0) OR " + "(state = 'completed' AND signing_key_version > 0 AND octet_length(lease_nonce_digest) = 32 AND lease_issued_at IS NOT NULL AND lease_expires_at > lease_issued_at AND lease_redeemed_at >= lease_issued_at AND failed_at IS NULL AND completed_at >= lease_redeemed_at AND resource_ref IS NOT NULL AND revision_id IS NOT NULL AND fragment_ref IS NOT NULL AND effect_count IN (0, 1))", + ) + + +def _restore_non_recoverable_fail_function() -> None: + op.execute( + f""" + CREATE OR REPLACE FUNCTION public.context_worker_fail_file_import( + requested_organization_id uuid, requested_job_id uuid, + requested_service_principal_id uuid, requested_source_ref text, + requested_signing_key_version bigint, requested_nonce bytea, + requested_issued_at timestamptz, + requested_expires_at timestamptz + ) RETURNS boolean + LANGUAGE plpgsql SECURITY DEFINER + SET search_path = pg_catalog, pg_temp SET row_security = on + AS $function$ + DECLARE changed boolean := false; failed_now timestamptz; + BEGIN + IF SESSION_USER <> '{_WORKER}' THEN RETURN false; END IF; + PERFORM pg_catalog.set_config( + 'app.organization_id', requested_organization_id::text, true + ); + PERFORM pg_catalog.set_config( + 'app.worker_job_id', requested_job_id::text, true + ); + failed_now := pg_catalog.statement_timestamp(); + UPDATE public.file_import_job AS job + SET state = 'failed', failed_at = failed_now + WHERE job.organization_id = requested_organization_id + AND job.job_id = requested_job_id + AND job.state = 'running' + AND job.service_principal_id = requested_service_principal_id + AND job.source_id::text = requested_source_ref + AND job.signing_key_version = requested_signing_key_version + AND job.lease_nonce_digest = public.digest(requested_nonce, 'sha256') + AND job.lease_issued_at = requested_issued_at + AND job.lease_expires_at = requested_expires_at + AND failed_now >= job.lease_issued_at + AND failed_now < job.lease_expires_at + AND EXISTS ( + SELECT 1 FROM public.service_principal AS principal + WHERE principal.organization_id = job.organization_id + AND principal.service_principal_id = job.service_principal_id + AND principal.workload = job.workload + AND principal.worker_audience = job.worker_audience + AND principal.operation = job.operation + AND principal.enabled IS TRUE + ); + changed := FOUND; + RETURN changed; + END; $function$ + """ + ) diff --git a/scripts/security_gate/rls.py b/scripts/security_gate/rls.py index cb27bb4f..dfb5d7dd 100644 --- a/scripts/security_gate/rls.py +++ b/scripts/security_gate/rls.py @@ -26,6 +26,8 @@ "file_acquisition": "PG-FILE-IMPORT-023", "file_acquisition_result": "PG-FILE-IMPORT-023", "file_import_job": "PG-FILE-IMPORT-023", + "file_import_job_event": "PG-FILE-RECOVERY-027", + "file_publication_recovery": "PG-FILE-RECOVERY-027", "file_resource_ingestion_guard": "PG-FILE-IMPORT-023", "file_revision_snapshot": "PG-FILE-IMPORT-023", "file_revision_replacement_plan": "PG-FILE-REPLACEMENT-026", diff --git a/tests/integration/test_file_import_tracer.py b/tests/integration/test_file_import_tracer.py index 83b5cb98..0eaa4371 100644 --- a/tests/integration/test_file_import_tracer.py +++ b/tests/integration/test_file_import_tracer.py @@ -558,7 +558,8 @@ def _redeem_direct( """ SELECT * FROM public.context_worker_redeem_file_import( :organization_id, :job_id, :service_principal_id, - :source_ref, :signing_key_version, :nonce, + :source_ref, :lease_generation, + :signing_key_version, :nonce, :issued_at, :expires_at ) """ @@ -570,6 +571,7 @@ def _redeem_direct( service_principal_id or claims.service_principal_id ), "source_ref": source_ref or claims.source_ref, + "lease_generation": claims.lease_generation, "signing_key_version": claims.signing_key_version, "nonce": claims.nonce, "issued_at": claims.issued_at, @@ -605,7 +607,8 @@ def _publish_direct( :content_hash, :compilation_digest, :compiler_version, :config_version, :phrase_digest, - :signing_key_version, :nonce, :issued_at, :expires_at + :lease_generation, :signing_key_version, :nonce, + :issued_at, :expires_at ) """ ), @@ -625,6 +628,7 @@ def _publish_direct( "phrase_digest": "c" * 64, "compiler_version": compiler_version, "config_version": config_version, + "lease_generation": claims.lease_generation, "signing_key_version": claims.signing_key_version, "nonce": claims.nonce, "issued_at": claims.issued_at, @@ -650,7 +654,8 @@ def _publish_structural_direct( :canonical_text, :content_hash, :compilation_digest, :compiler_version, :config_version, CAST(:compilation_document AS jsonb), - :signing_key_version, :nonce, :issued_at, :expires_at + :lease_generation, :signing_key_version, :nonce, + :issued_at, :expires_at ) """ ), @@ -667,6 +672,7 @@ def _publish_structural_direct( "compiler_version": document.provenance.compiler_version, "config_version": document.provenance.config_version, "compilation_document": json.dumps(compilation_document), + "lease_generation": claims.lease_generation, "signing_key_version": claims.signing_key_version, "nonce": claims.nonce, "issued_at": claims.issued_at, @@ -691,7 +697,8 @@ def _fail_direct( """ SELECT public.context_worker_fail_file_import( :organization_id, :job_id, :service_principal_id, - :source_ref, :signing_key_version, :nonce, + :source_ref, :lease_generation, + :signing_key_version, :nonce, :issued_at, :expires_at ) """ @@ -703,6 +710,7 @@ def _fail_direct( service_principal_id or claims.service_principal_id ), "source_ref": source_ref or claims.source_ref, + "lease_generation": claims.lease_generation, "signing_key_version": claims.signing_key_version, "nonce": claims.nonce, "issued_at": claims.issued_at, @@ -1411,7 +1419,7 @@ def _assert_structural_file_import_returns_coherent_authorized_units_over_http( connection.execute( text("SELECT version_num FROM alembic_version") ).scalar_one() - == "20260723_0014" + == "20260723_0015" ) diff --git a/tests/integration/test_m0_security_gate_rls.py b/tests/integration/test_m0_security_gate_rls.py index f25f03ff..defa1581 100644 --- a/tests/integration/test_m0_security_gate_rls.py +++ b/tests/integration/test_m0_security_gate_rls.py @@ -30,7 +30,7 @@ def _manifest() -> dict[str, object]: def test_all_manifest_tenant_tables_pass_live_non_owner_rls_audit( guarded_runtime_engine: Engine, ) -> None: - """PG-RLS-ALL-TENANT-TABLES: the live denominator is exactly 31/31.""" + """PG-RLS-ALL-TENANT-TABLES: the live denominator is exactly 33/33.""" with guarded_runtime_engine.connect() as connection: report = audit_live_rls( @@ -41,13 +41,13 @@ def test_all_manifest_tenant_tables_pass_live_non_owner_rls_audit( assert report["passed"] is True assert report["denominator"] == { - "allTables": 34, - "tenantOwned": 31, + "allTables": 36, + "tenantOwned": 33, "global": 3, } assert report["coverage"] == { - "numerator": 31, - "denominator": 31, + "numerator": 33, + "denominator": 33, "percent": 100.0, } assert report["failures"] == [] @@ -86,9 +86,9 @@ def test_no_force_row_level_security_mutation_fails_and_rolls_back( assert mutated["passed"] is False assert mutated["coverage"] == { - "numerator": 30, - "denominator": 31, - "percent": 96.77, + "numerator": 32, + "denominator": 33, + "percent": 96.97, } tenant_tables = cast(list[dict[str, Any]], mutated["tenantTables"]) organization_record = next( @@ -111,8 +111,8 @@ def test_no_force_row_level_security_mutation_fails_and_rolls_back( ) assert restored["passed"] is True assert restored["coverage"] == { - "numerator": 31, - "denominator": 31, + "numerator": 33, + "denominator": 33, "percent": 100.0, } diff --git a/tests/integration/test_migrations.py b/tests/integration/test_migrations.py index b94cffa7..b953658f 100644 --- a/tests/integration/test_migrations.py +++ b/tests/integration/test_migrations.py @@ -1,21 +1,41 @@ from __future__ import annotations +import json from concurrent.futures import ThreadPoolExecutor +from dataclasses import replace +from datetime import UTC, datetime from pathlib import Path from time import monotonic, sleep -from uuid import uuid4 +from uuid import UUID, uuid4 import pytest from alembic import command from alembic.config import Config -from sqlalchemy import text +from sqlalchemy import Engine, text from sqlalchemy.exc import IntegrityError, SQLAlchemyError -from engine.persistence import DatabaseConfiguration, create_database_engine +from adapters.parsers.markdown import compile_markdown +from engine.persistence import ( + DatabaseConfiguration, + PostgreSQLWorkerLeaseIssuer, + create_database_engine, +) +from engine.supply import ( + MarkdownCompilerConfig, + ParsedDocument, + canonicalize_parsed_document, +) from tests.integration.test_context_run_schema import ( LineageIdentity, insert_context_run, ) +from tests.integration.test_file_import_tracer import ( + _prepare_file_import_scenario, + _prepare_repeat_file_import, + _run_file_import, + _scenario_claims, +) +from tests.integration.test_zz_file_revision_replacement import NEW_MARKDOWN pytestmark = pytest.mark.integration ROOT = Path(__file__).parents[2] @@ -34,6 +54,8 @@ "file_acquisition", "file_acquisition_result", "file_import_job", + "file_import_job_event", + "file_publication_recovery", "file_resource_ingestion_guard", "file_revision_replacement_plan", "file_revision_snapshot", @@ -57,6 +79,82 @@ ] +def _delete_issue_27_upgrade_fixture( + configuration: DatabaseConfiguration, + organization_id: UUID, +) -> None: + """Remove only the disposable migration-compatibility scenario.""" + + engine = create_database_engine(configuration) + immutable_tables = ( + ("file_import_job_event", "file_import_job_event_immutable"), + ("file_revision_supersession", "file_revision_supersession_immutable"), + ("file_revision_replacement_plan", "file_revision_replacement_plan_immutable"), + ("exact_phrase_candidate", "exact_phrase_candidate_immutable"), + ("revision_publication_event", "revision_publication_event_immutable"), + ("context_fragment", "context_fragment_reject_mutation"), + ("file_revision_snapshot", "file_revision_snapshot_immutable"), + ("context_revision", "context_revision_reject_mutation"), + ("file_acquisition_result", "file_acquisition_result_immutable"), + ("file_resource_ingestion_guard", "file_resource_ingestion_guard_immutable"), + ("file_acquisition", "file_acquisition_immutable"), + ("source_version", "source_version_immutable"), + ) + try: + with engine.begin() as connection: + for table, trigger in immutable_tables: + connection.execute( + text(f"ALTER TABLE {table} DISABLE TRIGGER {trigger}") + ) + try: + with engine.begin() as connection: + for table in ( + "file_import_job_event", + "file_publication_recovery", + "file_revision_supersession", + "file_revision_replacement_plan", + "exact_phrase_candidate", + "revision_publication_event", + "membership_resource_field_right", + "resource_access_policy", + "context_fragment", + "file_revision_snapshot", + "context_revision", + "context_resource", + "file_acquisition_result", + "file_resource_ingestion_guard", + "file_import_job", + "file_acquisition", + "context_source", + "source_version", + "service_principal", + "membership", + ): + connection.execute( + text(f"DELETE FROM {table} WHERE organization_id = :org"), + {"org": organization_id}, + ) + connection.execute( + text( + "DELETE FROM user_account WHERE NOT EXISTS (" + "SELECT 1 FROM membership " + "WHERE membership.user_id = user_account.user_id)" + ) + ) + connection.execute( + text("DELETE FROM organization WHERE organization_id = :org"), + {"org": organization_id}, + ) + finally: + with engine.begin() as connection: + for table, trigger in reversed(immutable_tables): + connection.execute( + text(f"ALTER TABLE {table} ENABLE TRIGGER {trigger}") + ) + finally: + engine.dispose() + + def _revision_rows(configuration: DatabaseConfiguration) -> list[str]: engine = create_database_engine(configuration) try: @@ -103,7 +201,7 @@ def test_empty_baseline_remains_a_reversible_historical_revision( assert _application_tables(migration_configuration) == ["alembic_version"] finally: command.upgrade(alembic_configuration, "head") - assert _revision_rows(migration_configuration) == ["20260723_0014"] + assert _revision_rows(migration_configuration) == ["20260723_0015"] def test_organization_isolation_revision_downgrades_and_reapplies_cleanly( @@ -118,7 +216,7 @@ def test_organization_isolation_revision_downgrades_and_reapplies_cleanly( finally: command.upgrade(alembic_configuration, "head") - assert _revision_rows(migration_configuration) == ["20260723_0014"] + assert _revision_rows(migration_configuration) == ["20260723_0015"] assert _application_tables(migration_configuration) == HEAD_TABLES @@ -138,7 +236,7 @@ def test_membership_revision_downgrades_to_issue_8_and_reapplies_cleanly( finally: command.upgrade(alembic_configuration, "head") - assert _revision_rows(migration_configuration) == ["20260723_0014"] + assert _revision_rows(migration_configuration) == ["20260723_0015"] def test_content_schema_revision_downgrades_to_membership_and_reapplies_cleanly( @@ -159,7 +257,7 @@ def test_content_schema_revision_downgrades_to_membership_and_reapplies_cleanly( finally: command.upgrade(alembic_configuration, "head") - assert _revision_rows(migration_configuration) == ["20260723_0014"] + assert _revision_rows(migration_configuration) == ["20260723_0015"] assert _application_tables(migration_configuration) == HEAD_TABLES @@ -186,7 +284,7 @@ def test_policy_epoch_revision_downgrades_to_content_and_reapplies_cleanly( finally: command.upgrade(alembic_configuration, "head") - assert _revision_rows(migration_configuration) == ["20260723_0014"] + assert _revision_rows(migration_configuration) == ["20260723_0015"] assert _application_tables(migration_configuration) == HEAD_TABLES @@ -215,7 +313,7 @@ def test_worker_lease_revision_downgrades_to_policy_epoch_and_reapplies_cleanly( finally: command.upgrade(alembic_configuration, "head") - assert _revision_rows(migration_configuration) == ["20260723_0014"] + assert _revision_rows(migration_configuration) == ["20260723_0015"] assert _application_tables(migration_configuration) == HEAD_TABLES @@ -237,7 +335,7 @@ def test_decision_lineage_revision_downgrades_to_worker_lease_and_reapplies_clea finally: command.upgrade(alembic_configuration, "head") - assert _revision_rows(migration_configuration) == ["20260723_0014"] + assert _revision_rows(migration_configuration) == ["20260723_0015"] assert _application_tables(migration_configuration) == HEAD_TABLES @@ -275,7 +373,7 @@ def test_field_projection_revision_downgrades_to_decision_lineage_and_reapplies_ finally: command.upgrade(alembic_configuration, "head") - assert _revision_rows(migration_configuration) == ["20260723_0014"] + assert _revision_rows(migration_configuration) == ["20260723_0015"] assert "context_fragment_field" in _application_tables(migration_configuration) assert "membership_resource_field_right" in _application_tables( migration_configuration @@ -298,7 +396,7 @@ def test_file_source_revision_downgrades_to_learning_release_and_reapplies_clean finally: command.upgrade(alembic_configuration, "head") - assert _revision_rows(migration_configuration) == ["20260723_0014"] + assert _revision_rows(migration_configuration) == ["20260723_0015"] tables = _application_tables(migration_configuration) assert "context_source" in tables assert "source_version" in tables @@ -349,7 +447,7 @@ def test_structural_markdown_revision_downgrades_and_reapplies_cleanly( command.upgrade(alembic_configuration, "head") engine.dispose() - assert _revision_rows(migration_configuration) == ["20260723_0014"] + assert _revision_rows(migration_configuration) == ["20260723_0015"] engine = create_database_engine(migration_configuration) try: with engine.connect() as connection: @@ -402,7 +500,7 @@ def test_file_noop_revision_downgrades_and_reapplies_cleanly( finally: command.upgrade(alembic_configuration, "head") - assert _revision_rows(migration_configuration) == ["20260723_0014"] + assert _revision_rows(migration_configuration) == ["20260723_0015"] tables = _application_tables(migration_configuration) assert "file_acquisition_result" in tables assert "file_resource_ingestion_guard" in tables @@ -423,12 +521,173 @@ def test_file_replacement_revision_downgrades_and_reapplies_cleanly( finally: command.upgrade(alembic_configuration, "head") - assert _revision_rows(migration_configuration) == ["20260723_0014"] + assert _revision_rows(migration_configuration) == ["20260723_0015"] tables = _application_tables(migration_configuration) assert "file_revision_replacement_plan" in tables assert "file_revision_supersession" in tables +def test_file_recovery_revision_downgrades_and_reapplies_cleanly( + migration_configuration: DatabaseConfiguration, +) -> None: + """Issue #27 owns one reversible empty recovery schema boundary.""" + + alembic_configuration = Config(ROOT / "alembic.ini") + try: + command.downgrade(alembic_configuration, "20260723_0014") + assert _revision_rows(migration_configuration) == ["20260723_0014"] + tables = _application_tables(migration_configuration) + assert "file_publication_recovery" not in tables + assert "file_import_job_event" not in tables + finally: + command.upgrade(alembic_configuration, "head") + + assert _revision_rows(migration_configuration) == ["20260723_0015"] + tables = _application_tables(migration_configuration) + assert "file_publication_recovery" in tables + assert "file_import_job_event" in tables + + +def test_recovery_upgrade_adopts_an_existing_ready_replacement( + tmp_path: Path, + migration_configuration: DatabaseConfiguration, + guarded_control_engine: Engine, + guarded_worker_engine: Engine, +) -> None: + """An Issue #26 ready job remains resumable after the Issue #27 upgrade.""" + + scenario = _prepare_file_import_scenario( + tmp_path, + migration_configuration, + guarded_control_engine, + payload=b"# Handbook\n\nOLD marker.\n\n## Shared\n\nShared query.\n", + ) + assert scenario.token is not None + initial = _run_file_import( + scenario, + scenario.prepared, + scenario.token, + guarded_worker_engine, + config_version="markdown-config-v2", + ) + (scenario.root / "handbook.md").write_bytes(NEW_MARKDOWN) + replacement, replacement_token = _prepare_repeat_file_import( + scenario, + guarded_control_engine, + idempotency_key="ready-before-recovery-upgrade", + lease_ttl_seconds=2, + ) + claims = _scenario_claims( + replace(scenario, prepared=replacement, token=replacement_token) + ) + document = compile_markdown( + NEW_MARKDOWN, MarkdownCompilerConfig("markdown-config-v2") + ) + assert type(document) is ParsedDocument + revision_id = uuid4() + resource_ref = initial.candidate_ref.resource_ref + alembic_configuration = Config(ROOT / "alembic.ini") + try: + command.downgrade(alembic_configuration, "20260723_0014") + with guarded_worker_engine.begin() as connection: + redeemed = connection.execute( + text( + """ + SELECT * FROM public.context_worker_redeem_file_import( + :organization_id, :job_id, :service_principal_id, + :source_ref, :signing_key_version, :nonce, + :issued_at, :expires_at + ) + """ + ), + { + "organization_id": claims.organization_id, + "job_id": claims.job_id, + "service_principal_id": claims.service_principal_id, + "source_ref": claims.source_ref, + "signing_key_version": claims.signing_key_version, + "nonce": claims.nonce, + "issued_at": claims.issued_at, + "expires_at": claims.expires_at, + }, + ).one_or_none() + assert redeemed is not None + staged = connection.execute( + text( + """ + SELECT * + FROM public.context_worker_stage_structural_file_replacement( + :organization_id, :job_id, :service_principal_id, + :source_ref, :resource_ref, :revision_id, + :canonical_text, :content_hash, :compilation_digest, + :compiler_version, :config_version, + CAST(:compilation_document AS jsonb), + :signing_key_version, :nonce, :issued_at, :expires_at + ) + """ + ), + { + "organization_id": claims.organization_id, + "job_id": claims.job_id, + "service_principal_id": claims.service_principal_id, + "source_ref": claims.source_ref, + "resource_ref": resource_ref, + "revision_id": revision_id, + "canonical_text": document.canonical_text, + "content_hash": document.content_hash, + "compilation_digest": document.compilation_digest, + "compiler_version": document.provenance.compiler_version, + "config_version": document.provenance.config_version, + "compilation_document": json.dumps( + json.loads(canonicalize_parsed_document(document)), + ensure_ascii=False, + separators=(",", ":"), + ), + "signing_key_version": claims.signing_key_version, + "nonce": claims.nonce, + "issued_at": claims.issued_at, + "expires_at": claims.expires_at, + }, + ).one_or_none() + assert staged is not None + command.upgrade(alembic_configuration, "head") + migration_engine = create_database_engine(migration_configuration) + try: + with migration_engine.connect() as connection: + assert connection.execute( + text( + "SELECT checkpoint FROM file_publication_recovery " + "WHERE organization_id = :organization_id " + "AND job_id = :job_id" + ), + { + "organization_id": claims.organization_id, + "job_id": claims.job_id, + }, + ).scalar_one() == "ready" + finally: + migration_engine.dispose() + while datetime.now(UTC) <= claims.expires_at: + sleep(0.05) + recovered_token = PostgreSQLWorkerLeaseIssuer( + guarded_control_engine, scenario.codec + ).issue_file_import_lease(replacement) + recovered = _run_file_import( + scenario, + replacement, + recovered_token, + guarded_worker_engine, + config_version="markdown-config-v2", + ) + assert recovered.outcome == "replaced" + assert recovered.candidate_ref.revision_ref == str(revision_id) + finally: + command.upgrade(alembic_configuration, "head") + _delete_issue_27_upgrade_fixture( + migration_configuration, scenario.organization_id + ) + + def test_structural_snapshot_constraint_rejects_missing_json_bindings( migration_configuration: DatabaseConfiguration, ) -> None: @@ -706,7 +965,7 @@ def test_field_projection_downgrade_refuses_populated_content_atomically( ): command.downgrade(alembic_configuration, "20260722_0007") - assert _revision_rows(migration_configuration) == ["20260723_0014"] + assert _revision_rows(migration_configuration) == ["20260723_0015"] with engine.connect() as connection: assert ( connection.execute( @@ -772,7 +1031,7 @@ def test_field_projection_downgrade_refuses_populated_content_atomically( ): connection.execute(text(statement), parameters) except SQLAlchemyError: - if _revision_rows(migration_configuration) != ["20260723_0014"]: + if _revision_rows(migration_configuration) != ["20260723_0015"]: command.upgrade(alembic_configuration, "head") raise finally: @@ -926,7 +1185,7 @@ def test_field_projection_downgrade_serializes_with_concurrent_fragment_insert( == "concurrent-private-body" ) finally: - if _revision_rows(migration_configuration) != ["20260723_0014"]: + if _revision_rows(migration_configuration) != ["20260723_0015"]: command.upgrade(alembic_configuration, "head") with engine.begin() as connection: connection.execute( diff --git a/tests/integration/test_zz_file_content_noop.py b/tests/integration/test_zz_file_content_noop.py index 38487ad1..813bd5dc 100644 --- a/tests/integration/test_zz_file_content_noop.py +++ b/tests/integration/test_zz_file_content_noop.py @@ -17,6 +17,7 @@ from engine.control import FileImportPath from engine.persistence import ( DatabaseConfiguration, + FileImportUnavailable, PostgreSQLMembershipAuthority, create_database_engine, ) @@ -246,7 +247,7 @@ def test_repeated_canonically_identical_file_import_is_an_auditable_noop( with pytest.raises( RuntimeError, - match="File (?:replacement|no-op) downgrade requires", + match="File (?:recovery|replacement|no-op) downgrade requires", ): command.downgrade(Config(ROOT / "alembic.ini"), "20260723_0012") with migration_engine.connect() as connection: @@ -254,7 +255,7 @@ def test_repeated_canonically_identical_file_import_is_an_auditable_noop( connection.execute( text("SELECT version_num FROM alembic_version") ).scalar_one() - == "20260723_0014" + == "20260723_0015" ) @@ -439,6 +440,101 @@ def test_repeated_identical_structural_import_reuses_exact_active_artifact( migration_engine.dispose() +def test_extra_active_index_candidate_is_not_treated_as_unchanged( + tmp_path: Path, + migration_configuration: DatabaseConfiguration, + guarded_control_engine: Engine, + guarded_worker_engine: Engine, +) -> None: + scenario = _prepare_file_import_scenario( + tmp_path, + migration_configuration, + guarded_control_engine, + payload=(MARKDOWN_FIXTURES / "combined-v2.md").read_bytes(), + ) + assert scenario.token is not None + first = _run_file_import( + scenario, + scenario.prepared, + scenario.token, + guarded_worker_engine, + config_version="markdown-config-v2", + ) + repeat_prepared, repeat_token = _prepare_repeat_file_import( + scenario, + guarded_control_engine, + idempotency_key="repeat-corrupted-active-index", + ) + + migration_engine = create_database_engine(migration_configuration) + try: + with migration_engine.begin() as connection: + connection.execute( + text( + """ + INSERT INTO exact_phrase_candidate ( + organization_id, phrase_digest, source_ref, + resource_ref, revision_id, fragment_ref + ) VALUES ( + :organization_id, :phrase_digest, :source_ref, + :resource_ref, :revision_id, :fragment_ref + ) + """ + ), + { + "organization_id": scenario.organization_id, + "phrase_digest": sha256( + b"context-engine.exact-phrase.v1\x00unexpected phrase" + ).hexdigest(), + "source_ref": first.candidate_ref.source_ref, + "resource_ref": first.candidate_ref.resource_ref, + "revision_id": UUID(first.candidate_ref.revision_ref), + "fragment_ref": first.candidate_ref.fragment_ref, + }, + ) + + with pytest.raises(FileImportUnavailable): + _run_file_import( + scenario, + repeat_prepared, + repeat_token, + guarded_worker_engine, + config_version="markdown-config-v2", + ) + + with migration_engine.connect() as connection: + assert ( + connection.execute( + text( + """ + SELECT state FROM file_import_job + WHERE organization_id = :organization_id + AND job_id = :job_id + """ + ), + { + "organization_id": scenario.organization_id, + "job_id": repeat_prepared.job_id, + }, + ).scalar_one() + == "failed" + ) + assert ( + connection.execute( + text( + """ + SELECT count(*) FROM file_acquisition_result + WHERE organization_id = :organization_id + """ + ), + {"organization_id": scenario.organization_id}, + ).scalar_one() + == 0 + ) + finally: + migration_engine.dispose() + + @pytest.mark.parametrize( ("payload", "config_version"), ( diff --git a/tests/integration/test_zz_file_publication_recovery.py b/tests/integration/test_zz_file_publication_recovery.py new file mode 100644 index 00000000..d0ecb131 --- /dev/null +++ b/tests/integration/test_zz_file_publication_recovery.py @@ -0,0 +1,744 @@ +from __future__ import annotations + +import json +from concurrent.futures import ThreadPoolExecutor +from datetime import UTC, datetime +from pathlib import Path +from typing import Any, cast +from uuid import UUID + +import pytest +from sqlalchemy import Engine, text +from sqlalchemy.exc import SQLAlchemyError + +from adapters.file_source import FileReadLimits, FileRootRegistry +from adapters.parsers.markdown import compile_markdown +from engine.control import FileRootRef +from engine.persistence import ( + DatabaseConfiguration, + FileImportInterrupted, + FileImportLeaseRedemption, + FilePublicationBoundary, + PostgreSQLFileImportWorker, + PostgreSQLWorkerLeaseIssuer, + WorkerLeaseIssueNotAvailable, + create_database_engine, +) +from engine.supply import ( + MarkdownCompilerConfig, + ParsedDocument, + WorkNotAvailable, + canonicalize_parsed_document, +) +from tests.integration.test_file_import_tracer import ( + _FileImportScenario, + _prepare_file_import_scenario, + _prepare_repeat_file_import, + _redeem_direct, + _run_file_import, + _scenario_claims, +) +from tests.integration.test_zz_file_revision_replacement import ( + NEW_MARKDOWN, + NEW_V1_MARKDOWN, + OLD_MARKDOWN, + OLD_V1_MARKDOWN, + _resolve, + _scenario_user_id, +) + +pytestmark = pytest.mark.integration + + +def _worker( + scenario: _FileImportScenario, + guarded_worker_engine: Engine, + *, + config_version: str, + interrupt_after: FilePublicationBoundary | None = None, +) -> PostgreSQLFileImportWorker: + return PostgreSQLFileImportWorker( + guarded_worker_engine, + scenario.codec, + scenario.receiver, + FileRootRegistry( + {scenario.root_ref: scenario.root}, + limits=FileReadLimits(max_file_bytes=4096), + ), + MarkdownCompilerConfig(config_version), + clock=lambda: datetime.now(UTC).replace(microsecond=0), + interrupt_after=interrupt_after, + ) + + +def _run( + worker: PostgreSQLFileImportWorker, + scenario: _FileImportScenario, + token: object, +) -> object: + return worker.run( + FileImportLeaseRedemption( + cast(Any, token), + scenario.organization_id, + scenario.prepared.job_id, + scenario.source_ref, + ) + ) + + +def _wait_for_expiry(configuration: DatabaseConfiguration) -> None: + engine = create_database_engine(configuration) + try: + with engine.connect() as connection: + connection.execute(text("SELECT pg_sleep(2.1)")) + finally: + engine.dispose() + + +def _job_snapshot( + configuration: DatabaseConfiguration, + scenario: _FileImportScenario, +) -> tuple[object, ...]: + engine = create_database_engine(configuration) + try: + with engine.connect() as connection: + row = connection.execute( + text( + """ + SELECT job.state, job.lease_generation, + job.resource_ref, job.revision_id, + resource.active_revision_id, + (SELECT count(*) FROM context_revision AS revision + WHERE revision.organization_id = job.organization_id), + (SELECT count(*) FROM context_fragment AS fragment + WHERE fragment.organization_id = job.organization_id), + (SELECT count(*) FROM exact_phrase_candidate AS candidate + WHERE candidate.organization_id = job.organization_id), + (SELECT count(*) FROM file_import_job AS counted_job + WHERE counted_job.organization_id = job.organization_id), + (SELECT count(*) + FROM context_resource AS counted_resource + WHERE counted_resource.organization_id = + job.organization_id) + FROM file_import_job AS job + LEFT JOIN context_resource AS resource + ON resource.organization_id = job.organization_id + AND resource.resource_ref = job.resource_ref + WHERE job.organization_id = :organization_id + AND job.job_id = :job_id + """ + ), + { + "organization_id": scenario.organization_id, + "job_id": scenario.prepared.job_id, + }, + ).one() + return tuple(row) + finally: + engine.dispose() + + +def _history( + configuration: DatabaseConfiguration, + scenario: _FileImportScenario, +) -> tuple[tuple[object, ...], ...]: + engine = create_database_engine(configuration) + try: + with engine.connect() as connection: + rows = connection.execute( + text( + """ + SELECT event_type, boundary, lease_generation, + state_at_event, revision_id, + reason_digest IS NOT NULL + FROM file_import_job_event + WHERE organization_id = :organization_id + AND job_id = :job_id + ORDER BY ordinal + """ + ), + { + "organization_id": scenario.organization_id, + "job_id": scenario.prepared.job_id, + }, + ).all() + return tuple(tuple(row) for row in rows) + finally: + engine.dispose() + + +@pytest.mark.parametrize( + ( + "payload", + "config_version", + "expected_fragments", + "expected_candidates", + ), + [ + (OLD_V1_MARKDOWN, "markdown-config-v1", 1, 1), + (OLD_MARKDOWN, "markdown-config-v2", 4, 6), + ], +) +@pytest.mark.parametrize( + ("boundary", "interrupted_state", "pre_recovery_counts"), + [ + (FilePublicationBoundary.ACQUIRED, "running", (0, 0, 0)), + (FilePublicationBoundary.PREPARED, "prepared", None), + (FilePublicationBoundary.INDEXED, "ready", None), + ], +) +def test_each_durable_boundary_recovers_once_with_exact_lineage( + tmp_path: Path, + migration_configuration: DatabaseConfiguration, + guarded_control_engine: Engine, + guarded_worker_engine: Engine, + guarded_runtime_engine: Engine, + query_digest_keyring: object, + payload: bytes, + config_version: str, + expected_fragments: int, + expected_candidates: int, + boundary: FilePublicationBoundary, + interrupted_state: str, + pre_recovery_counts: tuple[int, int, int] | None, +) -> None: + scenario = _prepare_file_import_scenario( + tmp_path, + migration_configuration, + guarded_control_engine, + payload=payload, + lease_ttl_seconds=2, + ) + assert scenario.token is not None + old_claims = _scenario_claims(scenario) + + with pytest.raises(FileImportInterrupted) as interrupted: + _run( + _worker( + scenario, + guarded_worker_engine, + config_version=config_version, + interrupt_after=boundary, + ), + scenario, + scenario.token, + ) + assert interrupted.value.boundary is boundary + + before = _job_snapshot(migration_configuration, scenario) + assert before[0] == interrupted_state + assert before[1] == 1 + if pre_recovery_counts is not None: + assert tuple(before[5:8]) == pre_recovery_counts + assert tuple(before[8:]) == (1, 0) + elif boundary is FilePublicationBoundary.PREPARED: + assert tuple(before[5:]) == (1, expected_fragments, 0, 1, 1) + else: + assert tuple(before[5:]) == ( + 1, + expected_fragments, + expected_candidates, + 1, + 1, + ) + assert _history(migration_configuration, scenario)[-1][:2] == ( + "interrupted", + boundary.value, + ) + interrupted_revision_id = before[3] + user_id = _scenario_user_id(scenario, migration_configuration) + hidden = _resolve( + scenario, + guarded_runtime_engine, + cast(Any, query_digest_keyring), + user_id=user_id, + query="OLD marker.", + request_id=f"initial-{config_version}-{boundary.value}-hidden", + ) + assert hidden["blocks"] == [] + assert hidden["evidence"] == [] + + _wait_for_expiry(migration_configuration) + recovery_token = PostgreSQLWorkerLeaseIssuer( + guarded_control_engine, + scenario.codec, + ).issue_file_import_lease(scenario.prepared) + assert _redeem_direct(guarded_worker_engine, old_claims) is None + recovered = cast( + Any, + _run( + _worker( + scenario, + guarded_worker_engine, + config_version=config_version, + ), + scenario, + recovery_token, + ), + ) + + after = _job_snapshot(migration_configuration, scenario) + assert after[0] == "completed" + assert after[1] == 2 + assert after[2] == recovered.candidate_ref.resource_ref + assert after[3] == UUID(recovered.candidate_ref.revision_ref) + assert after[3] == interrupted_revision_id + assert after[4] == UUID(recovered.candidate_ref.revision_ref) + assert tuple(after[5:]) == ( + 1, + expected_fragments, + expected_candidates, + 1, + 1, + ) + assert len(recovered.candidate_refs) == expected_fragments + assert len({candidate.revision_ref for candidate in recovered.candidate_refs}) == 1 + history = _history(migration_configuration, scenario) + assert [row[0] for row in history].count("interrupted") == 1 + assert [row[0] for row in history].count("reclaimed") == 1 + assert [row[0] for row in history].count("active") == 1 + assert history[-1][0:4] == ( + "active", + "active", + 2, + "completed", + ) + package = _resolve( + scenario, + guarded_runtime_engine, + cast(Any, query_digest_keyring), + user_id=user_id, + query="OLD marker.", + request_id=f"initial-{config_version}-{boundary.value}-active", + ) + assert [block["text"] for block in package["blocks"]] == ( + ["OLD marker."] + if config_version == "markdown-config-v1" + else ["# Handbook\n\nOLD marker."] + ) + assert len(package["evidence"]) == 1 + assert package["evidence"][0]["revisionRef"] == ( + recovered.candidate_ref.revision_ref + ) + + +def _acquire_recovery_direct( + guarded_worker_engine: Engine, + scenario: _FileImportScenario, + *, + artifact_document: object, +) -> object | None: + claims = _scenario_claims(scenario) + document = compile_markdown( + OLD_MARKDOWN, + MarkdownCompilerConfig("markdown-config-v2"), + ) + assert type(document) is ParsedDocument + compilation_document = canonicalize_parsed_document(document).decode("utf-8") + with guarded_worker_engine.begin() as connection: + return connection.execute( + text( + """ + SELECT * FROM public.context_worker_acquire_file_publication( + :organization_id, :job_id, :service_principal_id, + :source_ref, :resource_ref, :revision_id, + :canonical_text, :content_hash, :compilation_digest, + :compiler_version, :config_version, + CAST(:compilation_document AS jsonb), + CAST(:artifact_document AS jsonb), + :lease_generation, :signing_key_version, :nonce, + :issued_at, :expires_at + ) + """ + ), + { + "organization_id": claims.organization_id, + "job_id": claims.job_id, + "service_principal_id": claims.service_principal_id, + "source_ref": claims.source_ref, + "resource_ref": "resource:malformed-recovery-artifact", + "revision_id": UUID("00000000-0000-0000-0000-000000000027"), + "canonical_text": document.canonical_text, + "content_hash": document.content_hash, + "compilation_digest": document.compilation_digest, + "compiler_version": document.provenance.compiler_version, + "config_version": document.provenance.config_version, + "compilation_document": compilation_document, + "artifact_document": json.dumps(artifact_document), + "lease_generation": claims.lease_generation, + "signing_key_version": claims.signing_key_version, + "nonce": claims.nonce, + "issued_at": claims.issued_at, + "expires_at": claims.expires_at, + }, + ).one_or_none() + + +def test_structural_recovery_rejects_artifact_not_derived_from_compilation( + tmp_path: Path, + migration_configuration: DatabaseConfiguration, + guarded_control_engine: Engine, + guarded_worker_engine: Engine, +) -> None: + scenario = _prepare_file_import_scenario( + tmp_path, + migration_configuration, + guarded_control_engine, + payload=OLD_MARKDOWN, + ) + assert scenario.token is not None + assert _redeem_direct(guarded_worker_engine, _scenario_claims(scenario)) is not None + assert ( + _acquire_recovery_direct( + guarded_worker_engine, + scenario, + artifact_document=[ + { + "fragmentRef": "fragment:paragraph:1", + "contextualText": "attacker supplied body", + "searchPhrases": ["attacker supplied phrase"], + } + ], + ) + is None + ) + snapshot = _job_snapshot(migration_configuration, scenario) + assert snapshot[0] == "running" + assert snapshot[2:5] == (None, None, None) + assert snapshot[5:] == (0, 0, 0, 1, 0) + + +def test_concurrent_reclaim_and_redemption_have_one_owner_and_one_effect( + tmp_path: Path, + migration_configuration: DatabaseConfiguration, + guarded_control_engine: Engine, + guarded_worker_engine: Engine, +) -> None: + scenario = _prepare_file_import_scenario( + tmp_path, + migration_configuration, + guarded_control_engine, + payload=OLD_MARKDOWN, + lease_ttl_seconds=2, + ) + assert scenario.token is not None + with pytest.raises(FileImportInterrupted): + _run( + _worker( + scenario, + guarded_worker_engine, + config_version="markdown-config-v2", + interrupt_after=FilePublicationBoundary.PREPARED, + ), + scenario, + scenario.token, + ) + _wait_for_expiry(migration_configuration) + + def reclaim() -> object: + return PostgreSQLWorkerLeaseIssuer( + guarded_control_engine, + scenario.codec, + ).issue_file_import_lease(scenario.prepared) + + with ThreadPoolExecutor(max_workers=2) as executor: + futures = (executor.submit(reclaim), executor.submit(reclaim)) + results: list[object] = [] + failures: list[type[BaseException]] = [] + for future in futures: + try: + results.append(future.result(timeout=5)) + except BaseException as error: + failures.append(type(error)) + assert len(results) == 1 + assert failures == [WorkerLeaseIssueNotAvailable] + recovery_token = results[0] + + def resume() -> object: + return _run( + _worker( + scenario, + guarded_worker_engine, + config_version="markdown-config-v2", + ), + scenario, + recovery_token, + ) + + with ThreadPoolExecutor(max_workers=2) as executor: + futures = (executor.submit(resume), executor.submit(resume)) + successes: list[object] = [] + rejections: list[type[BaseException]] = [] + for future in futures: + try: + successes.append(future.result(timeout=5)) + except BaseException as error: + rejections.append(type(error)) + assert len(successes) == 1 + assert rejections == [WorkNotAvailable] + assert _job_snapshot(migration_configuration, scenario)[5:] == (1, 4, 6, 1, 1) + assert [row[0] for row in _history(migration_configuration, scenario)].count( + "active" + ) == 1 + + +@pytest.mark.security_evidence(id="PG-FILE-RECOVERY-027", layer="postgres") +def test_recovery_tables_deny_direct_access_to_every_nonowner_role( + guarded_control_engine: Engine, + guarded_runtime_engine: Engine, + guarded_worker_engine: Engine, +) -> None: + statements = ( + "SELECT count(*) FROM {table}", + "INSERT INTO {table} DEFAULT VALUES", + "UPDATE {table} SET organization_id = organization_id WHERE false", + "DELETE FROM {table} WHERE false", + ) + for engine in ( + guarded_control_engine, + guarded_runtime_engine, + guarded_worker_engine, + ): + for table in ("file_publication_recovery", "file_import_job_event"): + for statement in statements: + with engine.connect() as connection: + transaction = connection.begin() + try: + with pytest.raises(SQLAlchemyError): + connection.execute(text(statement.format(table=table))) + finally: + transaction.rollback() + + +def test_other_organization_lease_cannot_recover_interrupted_job( + tmp_path: Path, + migration_configuration: DatabaseConfiguration, + guarded_control_engine: Engine, + guarded_worker_engine: Engine, +) -> None: + protected = _prepare_file_import_scenario( + tmp_path, + migration_configuration, + guarded_control_engine, + payload=OLD_MARKDOWN, + lease_ttl_seconds=2, + ) + other = _prepare_file_import_scenario( + tmp_path, + migration_configuration, + guarded_control_engine, + payload=NEW_MARKDOWN, + ) + assert protected.token is not None + assert other.token is not None + with pytest.raises(FileImportInterrupted): + _run( + _worker( + protected, + guarded_worker_engine, + config_version="markdown-config-v2", + interrupt_after=FilePublicationBoundary.PREPARED, + ), + protected, + protected.token, + ) + protected_before = _job_snapshot(migration_configuration, protected) + + other_claims = _scenario_claims(other) + assert _redeem_direct( + guarded_worker_engine, + other_claims, + organization_id=protected.organization_id, + job_id=protected.prepared.job_id, + source_ref=str(protected.source_ref.value), + ) is None + assert _job_snapshot(migration_configuration, protected) == protected_before + + +@pytest.mark.parametrize( + ("old_payload", "new_payload", "config_version"), + [ + (OLD_V1_MARKDOWN, NEW_V1_MARKDOWN, "markdown-config-v1"), + (OLD_MARKDOWN, NEW_MARKDOWN, "markdown-config-v2"), + ], +) +def test_ready_replacement_recovery_keeps_old_http_package_until_activation( + tmp_path: Path, + migration_configuration: DatabaseConfiguration, + guarded_control_engine: Engine, + guarded_worker_engine: Engine, + guarded_runtime_engine: Engine, + query_digest_keyring: object, + old_payload: bytes, + new_payload: bytes, + config_version: str, +) -> None: + scenario = _prepare_file_import_scenario( + tmp_path, + migration_configuration, + guarded_control_engine, + payload=old_payload, + ) + assert scenario.token is not None + first = _run_file_import( + scenario, + scenario.prepared, + scenario.token, + guarded_worker_engine, + config_version=config_version, + ) + (scenario.root / "handbook.md").write_bytes(new_payload) + prepared, token = _prepare_repeat_file_import( + scenario, + guarded_control_engine, + idempotency_key=f"recover-ready-{config_version}", + lease_ttl_seconds=2, + ) + recovery_scenario = _FileImportScenario( + organization_id=scenario.organization_id, + membership_id=scenario.membership_id, + receiver=scenario.receiver, + source_ref=scenario.source_ref, + prepared=prepared, + codec=scenario.codec, + token=token, + root_ref=FileRootRef(scenario.root_ref.value), + root=scenario.root, + ) + with pytest.raises(FileImportInterrupted): + _run( + _worker( + recovery_scenario, + guarded_worker_engine, + config_version=config_version, + interrupt_after=FilePublicationBoundary.INDEXED, + ), + recovery_scenario, + token, + ) + + user_id = _scenario_user_id(scenario, migration_configuration) + old_query = "OLD marker." + old_package = _resolve( + scenario, + guarded_runtime_engine, + cast(Any, query_digest_keyring), + user_id=user_id, + query=old_query, + request_id=f"ready-recovery-old-{config_version}", + ) + assert old_package["evidence"][0]["revisionRef"] == ( + first.candidate_ref.revision_ref + ) + assert len(old_package["blocks"]) == 1 + assert len(old_package["evidence"]) == 1 + hidden_new = _resolve( + scenario, + guarded_runtime_engine, + cast(Any, query_digest_keyring), + user_id=user_id, + query="NEW marker.", + request_id=f"ready-recovery-hidden-{config_version}", + ) + assert hidden_new["blocks"] == [] + assert hidden_new["evidence"] == [] + + _wait_for_expiry(migration_configuration) + recovery_token = PostgreSQLWorkerLeaseIssuer( + guarded_control_engine, + scenario.codec, + ).issue_file_import_lease(prepared) + recovered = cast( + Any, + _run( + _worker( + recovery_scenario, + guarded_worker_engine, + config_version=config_version, + ), + recovery_scenario, + recovery_token, + ), + ) + new_package = _resolve( + scenario, + guarded_runtime_engine, + cast(Any, query_digest_keyring), + user_id=user_id, + query="NEW marker.", + request_id=f"ready-recovery-new-{config_version}", + ) + assert new_package["evidence"][0]["revisionRef"] == ( + recovered.candidate_ref.revision_ref + ) + assert len(new_package["blocks"]) == 1 + assert len(new_package["evidence"]) == 1 + assert recovered.candidate_ref.revision_ref != first.candidate_ref.revision_ref + assert [row[0] for row in _history(migration_configuration, recovery_scenario)][ + -3: + ] == ["interrupted", "reclaimed", "active"] + + +def test_ready_initial_recovery_revalidates_current_audience_authority( + tmp_path: Path, + migration_configuration: DatabaseConfiguration, + guarded_control_engine: Engine, + guarded_worker_engine: Engine, +) -> None: + scenario = _prepare_file_import_scenario( + tmp_path, + migration_configuration, + guarded_control_engine, + payload=OLD_MARKDOWN, + lease_ttl_seconds=2, + ) + assert scenario.token is not None + with pytest.raises(FileImportInterrupted): + _run( + _worker( + scenario, + guarded_worker_engine, + config_version="markdown-config-v2", + interrupt_after=FilePublicationBoundary.INDEXED, + ), + scenario, + scenario.token, + ) + + engine = create_database_engine(migration_configuration) + try: + with engine.begin() as connection: + connection.execute( + text( + """ + UPDATE membership SET status = 'revoked' + WHERE organization_id = :organization_id + AND membership_id = :membership_id + """ + ), + { + "organization_id": scenario.organization_id, + "membership_id": scenario.membership_id, + }, + ) + finally: + engine.dispose() + + _wait_for_expiry(migration_configuration) + recovery_token = PostgreSQLWorkerLeaseIssuer( + guarded_control_engine, + scenario.codec, + ).issue_file_import_lease(scenario.prepared) + with pytest.raises(WorkNotAvailable): + _run( + _worker( + scenario, + guarded_worker_engine, + config_version="markdown-config-v2", + ), + scenario, + recovery_token, + ) + after = _job_snapshot(migration_configuration, scenario) + assert after[0] == "ready" + assert after[4] is None diff --git a/tests/integration/test_zz_file_revision_replacement.py b/tests/integration/test_zz_file_revision_replacement.py index 57f94274..b7e5d2f4 100644 --- a/tests/integration/test_zz_file_revision_replacement.py +++ b/tests/integration/test_zz_file_revision_replacement.py @@ -147,6 +147,7 @@ def _stage_replacement_direct( "compilation_digest": document.compilation_digest, "compiler_version": document.provenance.compiler_version, "config_version": document.provenance.config_version, + "lease_generation": claims.lease_generation, "signing_key_version": claims.signing_key_version, "nonce": claims.nonce, "issued_at": claims.issued_at, @@ -162,7 +163,8 @@ def _stage_replacement_direct( :canonical_text, :content_hash, :compilation_digest, :compiler_version, :config_version, CAST(:compilation_document AS jsonb), - :signing_key_version, :nonce, :issued_at, :expires_at + :lease_generation, :signing_key_version, :nonce, + :issued_at, :expires_at ) """ parameters["compilation_document"] = json.dumps( @@ -180,7 +182,8 @@ def _stage_replacement_direct( :fragment_ref, :canonical_text, :paragraph, :content_hash, :compilation_digest, :compiler_version, :config_version, :phrase_digest, - :signing_key_version, :nonce, :issued_at, :expires_at + :lease_generation, :signing_key_version, :nonce, + :issued_at, :expires_at ) """ parameters.update( @@ -211,6 +214,7 @@ def _activate_replacement_direct( "resource_ref": resource_ref, "previous_revision_id": previous_revision_id, "replacement_revision_id": replacement_revision_id, + "lease_generation": claims.lease_generation, "signing_key_version": claims.signing_key_version, "nonce": claims.nonce, "issued_at": claims.issued_at, @@ -224,7 +228,8 @@ def _activate_replacement_direct( SELECT * FROM public.context_worker_activate_file_replacement( :organization_id, :job_id, :service_principal_id, :source_ref, :resource_ref, :previous_revision_id, - :replacement_revision_id, :signing_key_version, + :replacement_revision_id, :lease_generation, + :signing_key_version, :nonce, :issued_at, :expires_at ) """ @@ -471,33 +476,27 @@ def test_changed_import_race_returns_the_late_job_as_a_successful_noop( ) late_reached_replace = Event() winner_completed = Event() - original_replace = PostgreSQLFileImportWorker._replace + original_execute_one = PostgreSQLFileImportWorker._execute_one - def pause_late_replace( + def pause_late_acquire( worker: PostgreSQLFileImportWorker, - claims: WorkerLeaseClaims, - resource_ref: str, - revision_id: UUID, - document: ParsedDocument, - payload: dict[str, object], - *, - structural: bool, + statement: str, + parameters: dict[str, object], ) -> object | None: - if claims.job_id == late_prepared.job_id: + if ( + parameters["job_id"] == late_prepared.job_id + and "context_worker_acquire_file_publication" in statement + ): late_reached_replace.set() if not winner_completed.wait(timeout=5): raise AssertionError("replacement race winner did not complete") - return original_replace( - worker, - claims, - resource_ref, - revision_id, - document, - payload, - structural=structural, - ) + return original_execute_one(worker, statement, parameters) - monkeypatch.setattr(PostgreSQLFileImportWorker, "_replace", pause_late_replace) + monkeypatch.setattr( + PostgreSQLFileImportWorker, + "_execute_one", + pause_late_acquire, + ) try: with ThreadPoolExecutor(max_workers=1) as executor: late_future = executor.submit( diff --git a/tests/unit/test_m0_rls_inventory.py b/tests/unit/test_m0_rls_inventory.py index bb58dd82..4897061f 100644 --- a/tests/unit/test_m0_rls_inventory.py +++ b/tests/unit/test_m0_rls_inventory.py @@ -27,6 +27,8 @@ "file_acquisition", "file_acquisition_result", "file_import_job", + "file_import_job_event", + "file_publication_recovery", "file_resource_ingestion_guard", "file_revision_snapshot", "file_revision_replacement_plan", @@ -128,7 +130,7 @@ def test_manifest_declares_exact_live_table_denominator_and_rls_evidence() -> No assert global_tables == GLOBAL_TABLES assert tenant_tables == TENANT_TABLES - assert len(tables) == 34 + assert len(tables) == 36 for name in sorted(GLOBAL_TABLES): rationale = tables[name]["classificationRationale"] @@ -151,8 +153,8 @@ def test_rls_auditor_requires_every_live_control_and_non_owner_evidence() -> Non assert report["passed"] is True assert report["coverage"] == { - "numerator": 31, - "denominator": 31, + "numerator": 33, + "denominator": 33, "percent": 100.0, } inventory = cast(dict[str, object], report["inventory"]) @@ -177,7 +179,7 @@ def test_rls_auditor_does_not_count_force_rls_or_evidence_gaps() -> None: assert report["passed"] is False assert report["coverage"] == { "numerator": 0, - "denominator": 31, + "denominator": 33, "percent": 0.0, } tenant_reports = cast(list[dict[str, Any]], report["tenantTables"]) diff --git a/tests/unit/test_schema_security_manifest.py b/tests/unit/test_schema_security_manifest.py index 180c7525..e86032a6 100644 --- a/tests/unit/test_schema_security_manifest.py +++ b/tests/unit/test_schema_security_manifest.py @@ -32,7 +32,7 @@ def test_manifest_classifies_the_exact_current_release_schema() -> None: document = manifest() tables = table_entries(document) - assert document["manifestVersion"] == "14.0.0" + assert document["manifestVersion"] == "15.0.0" assert set(tables) == { "active_release_manifest", "alembic_version", @@ -48,6 +48,8 @@ def test_manifest_classifies_the_exact_current_release_schema() -> None: "file_acquisition", "file_acquisition_result", "file_import_job", + "file_import_job_event", + "file_publication_recovery", "file_resource_ingestion_guard", "file_revision_snapshot", "file_revision_replacement_plan", @@ -97,6 +99,8 @@ def test_manifest_classifies_the_exact_current_release_schema() -> None: "file_acquisition", "file_acquisition_result", "file_import_job", + "file_import_job_event", + "file_publication_recovery", "file_resource_ingestion_guard", "file_revision_snapshot", "file_revision_replacement_plan", @@ -153,20 +157,21 @@ def test_issue_24_structural_markdown_contract_is_versioned_and_function_only() ], } structural_function = "EXECUTE context_worker_publish_structural_file_import_v2" - for table_name in ( - "file_import_job", - "file_revision_snapshot", - "revision_publication_event", - "exact_phrase_candidate", - ): - assert ( - structural_function - in entries[table_name]["permittedOperations"]["context_engine_worker"] - ) - assert ( - entries["context_fragment"]["permittedOperations"]["context_engine_worker"] - == [] + assert structural_function in entries["file_revision_snapshot"][ + "permittedOperations" + ]["context_engine_worker"] + recovery_steps = { + "EXECUTE context_worker_acquire_file_publication", + "EXECUTE context_worker_prepare_file_publication", + "EXECUTE context_worker_index_file_publication", + "EXECUTE context_worker_activate_recoverable_file_publication", + } + assert recovery_steps <= set( + entries["file_import_job"]["permittedOperations"]["context_engine_worker"] ) + assert entries["context_fragment"]["permittedOperations"][ + "context_engine_worker" + ] == ["EXECUTE context_worker_prepare_file_publication"] def test_issue_25_file_noop_contract_is_tenant_scoped_and_function_only() -> None: @@ -309,6 +314,75 @@ def test_issue_26_file_replacement_contract_is_staged_and_function_only() -> Non assert entry["permittedOperations"]["context_engine_runtime"] == [] +def test_issue_27_file_recovery_contract_is_generation_fenced_and_auditable() -> None: + document = manifest() + entries = table_entries(document) + operation = next( + value + for value in document["controlOperations"] + if value["name"] == "recover_file_publication" + ) + lease_issue = next( + value + for value in document["controlOperations"] + if value["name"] == "issue_file_import_lease" + ) + + assert operation["durableBoundaries"] == [ + "acquired", + "prepared", + "ready", + "completed", + ] + assert operation["idempotencyBinding"] == [ + "organization_id", + "job_id", + "source_id", + "resource_ref", + "revision_id", + "content_identity_digest", + "publication_payload_digest", + ] + assert "context_worker_issue_file_import_lease" not in operation[ + "databaseFunctions" + ] + assert lease_issue["atomicWrites"] == [ + "file_import_job", + "file_import_job_event", + ] + assert "higher lease generation" in operation["leaseReclaim"] + checkpoint = entries["file_publication_recovery"] + history = entries["file_import_job_event"] + for entry in (checkpoint, history): + assert entry["rowLevelSecurity"]["enabled"] is True + assert entry["rowLevelSecurity"]["forced"] is True + assert entry["functionOnlyMutation"]["directTableMutationAllowed"] is False + assert entry["permittedOperations"]["context_engine_runtime"] == [] + assert history["immutableRows"]["events"] == ["UPDATE", "DELETE"] + assert checkpoint["retention"]["sourceContent"] == "none" + + boundary_functions = { + "acquired": "context_worker_acquire_file_publication", + "prepared": "context_worker_prepare_file_publication", + "ready": "context_worker_index_file_publication", + "completed": "context_worker_activate_recoverable_file_publication", + } + for boundary, tables in operation["atomicWritesByBoundary"].items(): + database_function = boundary_functions[boundary] + for table_name in tables: + function_only = entries[table_name]["functionOnlyMutation"] + declared = function_only.get( + "databaseFunctions", [function_only.get("databaseFunction")] + ) + assert database_function in declared + assert ( + f"EXECUTE {database_function}" + in entries[table_name]["permittedOperations"][ + "context_engine_worker" + ] + ) + + def test_issue_21_file_source_manifest_is_closed_and_role_separated() -> None: entries = table_entries(manifest()) source = entries["context_source"] @@ -1153,11 +1227,19 @@ def test_content_manifest_preserves_lineage_visibility_and_immutability() -> Non "context_revision": ["INSERT"], "context_fragment": ["SELECT", "INSERT"], } + worker_operations = { + "context_resource": [ + "EXECUTE context_worker_prepare_file_publication", + "EXECUTE context_worker_activate_recoverable_file_publication", + ], + "context_revision": ["EXECUTE context_worker_prepare_file_publication"], + "context_fragment": ["EXECUTE context_worker_prepare_file_publication"], + } for entry in (resource, revision, fragment): assert entry["organizationColumn"] == "organization_id" assert entry["permittedOperations"] == { "context_engine_runtime": ["SELECT"], - "context_engine_worker": [], + "context_engine_worker": worker_operations[entry["name"]], "context_engine_worker_lease_definer": expected_definer_operations[ entry["name"] ], @@ -1333,6 +1415,9 @@ def test_content_manifest_preserves_lineage_visibility_and_immutability() -> Non "context_engine_worker": [], } if entry["name"] == "membership_resource_field_right": + expected_operations["context_engine_worker"] = [ + "EXECUTE context_worker_prepare_file_publication" + ] expected_operations["context_engine_worker_lease_definer"] = [ "SELECT", "INSERT", @@ -1456,6 +1541,9 @@ def test_policy_epoch_manifest_seals_runtime_reads_and_control_mutation() -> Non "context_engine_worker": [], } if entry["name"] == "resource_access_policy": + expected_operations["context_engine_worker"] = [ + "EXECUTE context_worker_prepare_file_publication" + ] expected_operations["context_engine_worker_lease_definer"] = [ "SELECT", "INSERT", diff --git a/tests/unit/test_worker_lease.py b/tests/unit/test_worker_lease.py index 5418b20c..bc24b544 100644 --- a/tests/unit/test_worker_lease.py +++ b/tests/unit/test_worker_lease.py @@ -236,14 +236,16 @@ def test_file_import_lease_uses_a_distinct_version_and_exact_source_binding() -> workload="supply.file-import", operation=FILE_IMPORT_WORKER_LEASE_OPERATION, source_ref="source:handbook", + lease_generation=1, ) ) header, payload = _decoded_token(token) - assert header["v"] == 2 + assert header["v"] == 3 assert payload["operation"] == "file.import" assert payload["source_ref"] == "source:handbook" + assert payload["lease_generation"] == 1 assert codec.verify( token, **_verification_arguments( @@ -261,6 +263,7 @@ def test_file_import_lease_rejects_a_wrong_source_generically() -> None: workload="supply.file-import", operation=FILE_IMPORT_WORKER_LEASE_OPERATION, source_ref="source:handbook", + lease_generation=1, ) )