Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 33 additions & 1 deletion CONTEXT.md
Original file line number Diff line number Diff line change
Expand Up @@ -329,7 +329,9 @@ registered service workload,并防 cross-job/cross-tenant replay。
- **Owner/scope:** one Organization, one durable job attempt, its declared work,
and the registered service workload that may perform it.
- **Lifecycle:** short-lived and one-shot; mismatch, expiry, staleness, or replay
makes it invalid. Exact claim/redemption fields belong to the owning ADR.
makes it invalid. A reclaimed job receives a higher signed lease generation,
so the replaced generation and nonce cannot resume or mutate it. Exact
claim/redemption fields belong to the owning ADR.
- **Invariant:** no general tenant/read/action authority and no long-lived source
credential; rejected lease produces zero business effect.
- **Activation note:** Issue #17 binds a registered ServicePrincipal to
Expand Down Expand Up @@ -372,6 +374,36 @@ acquisition 精确绑定。
- **Do not confuse with:** active ContextRevision, publish watermark,
acquisition checkpoint, or recovery lease.

### `File publication recovery checkpoint`

The mutable durable step marker for one interrupted File publication. 中文:File
publication recovery checkpoint 将同一个 job 固定到唯一 Resource、Revision、内容
identity,并允许新租约从已提交边界继续。

- **Owner/scope:** one Organization, File import job, ContextSource,
ContextResource, and stable target ContextRevision.
- **Lifecycle:** advances `acquired -> prepared -> ready -> completed`; an
expired attempt may be reclaimed only by a higher signed lease generation.
- **Invariant:** it neither authorizes delivery nor makes a Revision active;
every resumed step and activation revalidates the current lease generation,
exact compilation/write-artifact digest, and current authority.
- **Do not confuse with:** File acquisition outcome, replacement plan,
publication event, active pointer, or automatic retry scheduler.

### `File import job event`

The immutable ordered audit of File publication boundaries and recovery. 中文:
File import job event 记录中断、租约回收及最终激活,且不保存源内容。

- **Owner/scope:** one Organization and File import job, with lease generation
and optional stable Revision lineage.
- **Lifecycle:** append-only events for acquired, prepared, indexed,
interrupted, reclaimed, unchanged, and active transitions.
- **Invariant:** interruption/reclaim reasons are fixed-category digests; the
event is audit evidence, never execution or content authority.
- **Do not confuse with:** Revision publication event, WorkerLease, queue
delivery, checkpoint, or retry policy.

### `File revision supersession`

The immutable old-to-new Revision edge recorded by successful File activation.
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
---
name: adr-0041-recover-file-publication-by-durable-boundary
version: "1.0.0"
description: >
Recover one interrupted File publication from stable acquired, prepared, or
ready state using a higher-generation lease and immutable transition audit.
---

# 0041. Recover File publication by durable boundary

- Status: accepted
- Date: 2026-07-23
- Refines: ADR-0029, ADR-0037, ADR-0039, ADR-0040

## Context

File publication now spans classification, immutable Revision construction,
candidate/index preparation, and the active-pointer transaction. A worker may
stop after any committed step. Deleting partial state and creating a new job
would lose the exact attempt identity, can duplicate immutable lineage, and can
create an empty visibility window during replacement.

Recovery must reuse the File job and WorkerLease protocol. It must also make an
expired lease permanently stale once a new worker owns the job, while keeping
the old active Revision available until activation.

## Decision

One File job has four durable checkpoints: `acquired`, `prepared`, `ready`, and
`completed`. `file_publication_recovery` binds the job to Organization, Source,
stable Resource, one generated Revision, optional previous Revision, versioned
content identity, compiler contract, and a digest of the exact compilation and
write artifacts. Structural-v2 artifacts must exactly project the validated
compilation document; v1 artifacts must exactly project the canonical paragraph.
Each step is a separate PostgreSQL transaction and advances only from its exact
predecessor.

Lease issue may reclaim an expired `leased`, `running`, `prepared`, or `ready`
job. It increments `lease_generation`, replaces the nonce/time binding, and
records the state to resume. The generation is a signed WorkerLease claim;
redemption restores that state and clears the transient resume marker. Every
later mutation, including compatibility publication seams, rechecks the current
exact generation and lease; an old nonce or generation can perform no work.

`file_import_job_event` is immutable and records acquired, prepared, indexed,
explicit interruption, reclaim, unchanged, and active transitions. Failure and
reclaim evidence uses fixed categories plus digests; it stores no source
content. Deterministic test interruption is injected only after a committed
boundary and is never auto-retried.

The Organization/Source/Resource ingestion guard serializes initial and changed
classification. A concurrent equivalent winner is re-observed as an auditable
zero-effect no-op. A second worker cannot redeem the same lease. Recovery
reuses the stable Revision and existing rows rather than inserting them again.

Activation revalidates the current ServicePrincipal, acquisition Membership,
body access, complete Fragment/candidate representation, and ordered
publication evidence. Initial activation changes a null active pointer once;
replacement delegates to ADR-0040's previous-to-new compare-and-swap. Therefore
Runtime sees the complete old Revision until the complete recovered Revision is
activated.

## Consequences

- The three named interruption boundaries resume to one active Revision without
duplicate Resource, Revision, Fragment, candidate, job, or Package evidence.
- A durable checkpoint is retained for later reclaim; pre-checkpoint authority
or compilation failures remain terminal and content-free.
- Recovery rows and job history are tenant-owned, FORCE-RLS protected, and
accessible to the worker only through SECURITY DEFINER functions.
- A recovery schema with any non-completed checkpoint or job event attached to
a non-completed job is intentionally non-downgradable because removing it
would erase the only resume contract or audit. Completed checkpoint/audit
metadata may be removed by an explicit schema downgrade because the immutable
publication lineage remains in the Issue #26 tables.
- Arbitrary instruction-level chaos, batch recovery, delete/tombstone recovery,
dead-letter handling, and operator remediation remain inactive.

## Revisit trigger

Revisit before automatic retry scheduling, explicit lease release, batch or
delete recovery, checkpoint compaction, dead-letter processing, or physical
cleanup. Any change must preserve exact Organization/job/Revision identity,
higher-generation lease fencing, audit continuity, and all-old/all-new Runtime
visibility.
1 change: 1 addition & 0 deletions docs/decisions/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -127,3 +127,4 @@ touched:
- [0038 — Structural Markdown units](0038-compile-and-publish-structural-markdown.md)
- [0039 — File acquisition no-op](0039-deduplicate-unchanged-file-acquisitions.md)
- [0040 — Atomic File replacement](0040-stage-and-atomically-activate-file-replacements.md)
- [0041 — Durable File publication recovery](0041-recover-file-publication-by-durable-boundary.md)
20 changes: 16 additions & 4 deletions docs/design/2026-07-18-context-engine-implementation-design.md
Original file line number Diff line number Diff line change
Expand Up @@ -561,14 +561,27 @@ durable replacement plan before marking the job `ready`. The old Revision stays
active throughout staging. A second transaction revalidates authority and
readiness, compare-and-swaps the Resource active pointer, appends `active`,
records immutable supersession lineage, and completes the job. Superseded
artifacts remain `retained_until_explicit_cleanup`; deletion and recovery of a
committed ready job are not active in this slice. When an equivalent concurrent
artifacts remain `retained_until_explicit_cleanup`; deletion is not active in
this slice. When an equivalent concurrent
replacement activates between the initial publish attempt and replacement
staging, the guarded stage classification completes the later job as
`unchanged` and returns that durable zero-effect result. V1 and V2 each reprove
that the supplied compilation exactly matches the now-active snapshot,
Fragments, and candidates before reporting success.

ADR-0041 makes publication recoverable after exactly the committed `acquired`,
`prepared`, and `ready` boundaries. One tenant-owned checkpoint binds the
existing File job to its stable Resource/Revision/content identity, while an
immutable job-event stream records interruption, reclaim, and completion.
Reclaim issues a higher signed lease generation only after expiry; the replaced
generation and nonce cannot resume or mutate, including through compatibility
publication seams. Each idempotent step revalidates the exact current lease and
advances one checkpoint. Initial and replacement activation recheck
current audience authority and complete Fragment/candidate evidence before the
single pointer transaction, so a recovered replacement leaves old active
content visible until the new Revision is complete. Automatic retry scheduling,
arbitrary chaos, delete recovery, and dead-letter handling remain inactive.

Because Runtime resolves through multiple SQL statements at `READ COMMITTED`,
each UserActor transaction takes an Organization-scoped shared publication
barrier and activation takes the matching exclusive transaction barrier around
Expand Down Expand Up @@ -832,8 +845,7 @@ versioned public wire and activates breaking-change checks.
Exit:

- Markdown AST, hash incrementality, delete detection, checkpoint replay,
publication crash recovery, lease reclaim, dead-letter, and full-resync
runbook are complete;
broader crash recovery, dead-letter, and full-resync runbook are complete;
- before experiment execution, the frozen dataset registers a sample plan based
on failure-slice coverage and a declared uncertainty/power target; negative
cases cover every active refusal/security category, and an underpowered set is
Expand Down
4 changes: 4 additions & 0 deletions engine/persistence/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -34,8 +34,10 @@
from engine.persistence.control_sources import PostgreSQLControlStore
from engine.persistence.database import create_database_engine
from engine.persistence.file_imports import (
FileImportInterrupted,
FileImportLeaseRedemption,
FileImportUnavailable,
FilePublicationBoundary,
PostgreSQLFileImportWorker,
PublishedFileImport,
)
Expand Down Expand Up @@ -99,7 +101,9 @@
"PostgreSQLContextRunReader",
"PostgreSQLControlStore",
"FileImportLeaseRedemption",
"FileImportInterrupted",
"FileImportUnavailable",
"FilePublicationBoundary",
"PostgreSQLFileImportWorker",
"PublishedFileImport",
"VerifiedContextRunOperatorIdentity",
Expand Down
Loading
Loading