Skip to content
Merged
Show file tree
Hide file tree
Changes from 14 commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
06b13b8
test(tool-capability): require lifecycle operability evidence
seonghobae Sep 9, 2026
9e5ea2b
feat(tool-capability): evaluate lifecycle operability evidence
seonghobae Sep 9, 2026
9c19278
docs(tool-capability): define lifecycle operability evidence
seonghobae Sep 9, 2026
be98bc3
feat(tool-capability): add lifecycle operability audit CLI
seonghobae Sep 9, 2026
c21a24d
test(tool-capability): cover lifecycle operability CLI
seonghobae Sep 9, 2026
e341a35
test(tool-capability): correct collection failure oracle
seonghobae Sep 9, 2026
db45017
chore(tool-capability): expose lifecycle operability audit
seonghobae Sep 9, 2026
5a419f5
fix(tool-capability): keep evidence path out of audit output
seonghobae Sep 9, 2026
bed238c
test(tool-capability): pin pathname non-retention
seonghobae Sep 9, 2026
a6b43fc
docs(tool-capability): bound lifecycle evidence claim
seonghobae Sep 9, 2026
36e5d9c
test(tool-capability): enforce lifecycle evidence coverage
seonghobae Sep 9, 2026
974f7c3
test(tool-capability): cover lifecycle evidence edge cases
seonghobae Sep 9, 2026
77bb444
test(tool-capability): cover lifecycle audit runtime defaults
seonghobae Sep 9, 2026
093f3f7
docs(tool-capability): bind lifecycle storage evidence to databaseSize
seonghobae Sep 9, 2026
9577dcb
test(tool-capability): bind contention latency denominator
seonghobae Sep 9, 2026
0f52f87
fix(tool-capability): bind contention p95 to CAS trials
seonghobae Sep 9, 2026
b30148b
test(tool-capability): align audit contention denominator
seonghobae Sep 9, 2026
4c2ca6a
test(tool-capability): align contention evidence fixtures
seonghobae Sep 9, 2026
292eec4
fix(test): restore lifecycle binding fixture
seonghobae Sep 9, 2026
04c21d4
test(tool-capability): cover contention sample authority
seonghobae Sep 9, 2026
b7c1804
docs(tool-capability): bind contention latency to CAS trials
seonghobae Sep 9, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# External-extension lifecycle operability evidence

## Decision status

This note defines the evidence gate for issue #561 after lifecycle source integration. It does not declare the lifecycle Durable Object deployed, does not authenticate a future evidence producer, and does not move AppGuardrail, quarantine-sandbox-runtime, EgressWeave, Keyverse, or contextual-orchestrator authority into Noema.

## Problem

The protected source can prove lifecycle transition legality, replay/CAS behavior, compact current projection verification, complete audit verification, and fresh activation-evidence checks in repository tests. Those tests cannot establish production Durable Object latency, contention behavior, storage growth, restart recovery, or rollback recovery. Accepting a local `workerd` run, a synthetic benchmark, a precomputed percentile, or a reduced sample set as production evidence would collapse that distinction.

## Platform constraints

Cloudflare recommends SQLite-backed Durable Objects for new namespaces and, as of 2026-07-09, new namespaces must use SQLite on accounts that do not already have a legacy KV-backed namespace. SQLite-backed objects retain the KV storage API and add SQL plus point-in-time recovery. The SQLite Storage API documents atomic/isolated storage operations, asynchronous `transaction()`, and SQLite-only synchronous storage/`transactionSync()` capabilities. Cloudflare also documents a 10 GB per-object SQLite storage limit and notes that a single Durable Object is inherently single-threaded, so contention and storage-growth evidence must be measured on the deployed object rather than inferred from unit-test concurrency.

The same SQLite Storage API exposes `ctx.storage.sql.databaseSize`, the current database size in bytes. That is the canonical source for this acceptance contract's per-object `bytes_before` / `bytes_after` observations. Cloudflare's 2026-07-20 namespace-level Total storage dashboard is useful operational corroboration, but Cloudflare explicitly does not expose that dashboard metric by individual Durable Object ID/name; therefore namespace chart values cannot substitute for the exact object's `databaseSize` evidence.

Noema already uses declarative Wrangler `exports` for its existing SQLite Durable Object classes. A later runtime-wiring change for the lifecycle class must therefore be reviewed as an explicit binding/export reconciliation and must not infer deployment from this evidence evaluator.

## Evidence contract

`scripts/lib/external-extension-lifecycle-operability-evidence.mjs` accepts only `source_kind = cloudflare_durable_object_remote` and `storage_backend = sqlite`. The evidence binds repository, protected/deployed revision, canonical lifecycle binding name, and a canonical UTC observation instant.

For both `read_current` and `contended_append`, the producer retains the full measured latency array, the planned denominator, failure count, and excluded warm-up count. Acceptance requires at least 100 retained measurements, zero excluded warm-up attempts, denominator equality, and zero failed requests. The evaluator calculates p95 itself using the deterministic nearest-rank rule and requires p95 <= 20 ms. A producer-supplied p95 is not authority.

Pairwise contention rehearsal requires one accepted CAS winner and one conflict loser per trial. Recovery evidence must retain at least 129 lifecycle events so the test crosses the Workflow / Task receipt-ring size, verify complete audit rebuild, verify restart recovery and rollback recovery, and demonstrate rejection of malformed head and truncated audit evidence. The remote producer reads `ctx.storage.sql.databaseSize` immediately before and after the same acceptance run so per-object storage growth is observable rather than inferred from a namespace aggregate.

These checks make structurally incomplete evidence and evidence explicitly labeled as local, synthetic, or fixture data fail closed. They do not detect a producer that lies about those fields and do not cryptographically authenticate the producer. The eventual deployment/evidence collector must bind these bytes to the exact deployed revision and runtime identity through the owning release/deployment path. Until that exists, a PASS from the pure evaluator is necessary but not sufficient for issue #561 completion.

## Alternatives rejected

- **Use local/workerd timing as production evidence.** Rejected because scheduler, storage, network, and deployed-object contention differ from the remote service.
- **Accept a reported p95.** Rejected because excluded failures, discarded tail samples, or warm-up filtering could make an unverifiable percentile look compliant.
- **Use namespace-level storage charts as per-object evidence.** Rejected because Cloudflare's dashboard metric is namespace-wide and cannot be viewed by Durable Object ID/name; the deployed object's `ctx.storage.sql.databaseSize` is the reviewed per-object source.
- **Scan the complete lifecycle audit on every latency-sensitive current read.** Rejected because `readCurrent()` intentionally verifies the compact head and exact tail in O(1) retained-event cardinality; full-prefix verification belongs to audit/recovery.
- **Copy foreign-owner verdicts into the benchmark.** Rejected because Noema consumes immutable owner references/digests and must not become the scanner, quarantine, or outbound-policy authority.

## Follow-up

1. Add the lifecycle Durable Object runtime binding/export only with a reviewed Noema-owned authority adapter for activation-time Policy / Approval and foreign-owner evidence resolution.
2. Produce remote evidence on an exact deployed revision without sample reduction or warm-up exclusion, reading per-object storage bytes from `ctx.storage.sql.databaseSize`.
3. Run the evaluator and retain the bounded evidence plus deployment/release provenance.
4. Exercise SQLite point-in-time recovery or an equivalent reviewed rollback path against the lifecycle object and retain the recovery receipt.
5. Keep ADR 0015 Proposed until runtime and release acceptance are complete.

## References

Cloudflare. (2026, June 30). *Declarative class lifecycle with `exports`*. Durable Objects release notes. https://developers.cloudflare.com/durable-objects/release-notes/

Cloudflare. (2026, July 9). *New Durable Object namespaces must use the SQLite storage backend*. Cloudflare changelog. https://developers.cloudflare.com/changelog/post/2026-07-09-restrict-new-kv-backed-namespaces/

Cloudflare. (2026, July 20). *View total SQLite storage for Durable Object namespaces*. Cloudflare changelog. https://developers.cloudflare.com/changelog/post/2026-07-20-durable-objects-total-storage-metrics/

Cloudflare. (2026, May 27). *SQLite-backed Durable Object Storage*. https://developers.cloudflare.com/durable-objects/api/sqlite-storage-api/

Cloudflare. (2026, June 1). *Limits*. Durable Objects. https://developers.cloudflare.com/durable-objects/platform/limits/
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@
"operations:workflow-registry-disable": "node scripts/workflow-registry-live-disable.mjs",
"operations:runner-assignment": "node scripts/actions-runner-assignment-audit.mjs",
"operations:external-scheduler-evidence": "node scripts/external-scheduler-evidence-audit.mjs",
"operations:lifecycle-operability": "node scripts/external-extension-lifecycle-operability-audit.mjs",
"production:governance": "node scripts/production-environment-governance-audit.mjs",
"production:preflight": "node scripts/production-evidence-preflight.mjs",
"patch-validator:image:verify-receipts": "node scripts/verify-patch-validator-image.mjs",
Expand Down
64 changes: 64 additions & 0 deletions scripts/external-extension-lifecycle-operability-audit.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
#!/usr/bin/env node
import { pathToFileURL } from "node:url";
import { evaluateExternalExtensionLifecycleOperabilityEvidence } from "./lib/external-extension-lifecycle-operability-evidence.mjs";
import { readStrictJsonEvidence } from "./lib/strict-json-evidence.mjs";

const DEFAULT_EVIDENCE_PATH = "external-extension-lifecycle-operability-evidence.json";

function collectionFailure(reason) {
return {
status: "FAIL",
checks: [{ code: "evidence_collection", pass: false }],
failures: [{
code: "evidence_collection",
detail: `lifecycle operability evidence could not be read safely (${reason})`,
}],
metrics: {
read_current_p95_ms: null,
contended_append_p95_ms: null,
storage_growth_bytes: null,
},
};
}

/** Resolve one explicit evidence path while preserving a deterministic operator default. */
export function resolveEvidencePath(argv) {
const raw = argv[2];
if (raw === undefined) return DEFAULT_EVIDENCE_PATH;
const normalized = String(raw).trim();
return normalized.length === 0 ? DEFAULT_EVIDENCE_PATH : normalized;
}

/**
* Evaluate one descriptor-safe retained evidence file and emit only bounded validation results.
* Raw samples, source evidence, and the caller-supplied evidence pathname are never echoed.
*/
export function main(options = {}) {
const argv = options.argv ?? process.argv;
const readEvidence = options.readEvidence ?? readStrictJsonEvidence;
const writeOutput = options.writeOutput ?? ((value) => process.stdout.write(value));
const setExitCode = options.setExitCode ?? ((code) => {
process.exitCode = code;
});
const evidencePath = resolveEvidencePath(argv);
const retained = readEvidence(evidencePath);
const result = retained.ok
? evaluateExternalExtensionLifecycleOperabilityEvidence(retained.value)
: collectionFailure(retained.reason);

writeOutput(`${JSON.stringify({
source: "external-extension-lifecycle-operability-audit",
...result,
})}\n`);
if (result.status !== "PASS") setExitCode(1);
return result;
}

/** Execute the CLI only when this module is the process entry point. */
export function runIfDirect(metaUrl, argv, execute) {
if (!argv[1] || metaUrl !== pathToFileURL(argv[1]).href) return false;
execute();
return true;
}

runIfDirect(import.meta.url, process.argv, main);
Loading
Loading