feat(tool-capability): verify lifecycle operability evidence

[seonghobae]

Scope

Add executable acceptance evidence for Noema external-extension lifecycle operability without claiming foreign-owner truth or synthetic runtime proof. This lane owns only the evidence schema/evaluator and its tests. It does not add provider routing, AppGuardrail/quarantine/Egress authority, or fabricate Durable Object deployment evidence.

RED → minimal causal repair

The original test-only exact 06b13b863c20799963e9d011d5eb613372b902dc required an evaluator that did not exist. Implementation reached 974f7c3650e98bc03d51091f5aaf5467c4608b01; hosted application CI 34383704092 checked out that exact head against protected main@73039b556338e96df591b61cf8992a41c952ef35. All 636 test files / 4,327 tests passed, but the 100% owned-production coverage gate remained RED because the audit CLI's production dependency defaults (argv, strict reader, stdout writer, exit-code setter) were not executed. 77bb444727275d4114fe10ef49c31ab16946e9bd added runtime-default coverage without excluding source or weakening the threshold.

A subsequent current-head review found a distinct evidence-integrity gap: contended_append.latency_ms was not structurally bound to contention_trials, so a small pairwise CAS proof could lend its contended label to an unrelated low-latency sample series. Test-only exact 9577dcbe96714c59852541c1d824ba28e873f19f made that finding a reality RED. Hosted CI 34387316161 / job 102586750773 passed exact checkout, live-base validation, lock control, install and typecheck, then failed exactly one new regression: 635 files / 4,328 tests passed and the hostile test failed because contention_sample_denominator did not yet exist.

Current exact b7c18041a53dca2184ff4e85d8201063cf87ccad applies the minimum causal repair: pairwise contention must map to exactly two measured append attempts per trial (planned_samples = 2 * contention_trials), with safe-integer bounds; passing fixtures now use 50 trials / 100 measured attempts, overflow and denominator drift fail closed, and the doctoring note records the same evidence semantics. p95, failure-denominator, CAS-winner/loser and recovery requirements were not relaxed. All predecessor GREEN is invalidated; only fresh checks on this exact head may authorize readiness or merge.

Acceptance

• evidence source must be a real remote Cloudflare Durable Object observation, never local/workerd/synthetic/fixture;
• raw retained latency samples must preserve the planned denominator with no excluded warm-up samples or failed-request omission;
• evaluator computes deterministic p95 itself and requires current-read and contended-append p95 <= 20 ms;
• pairwise contention must prove exactly one winner and one conflict loser per trial, and the contended latency denominator must contain both attempts from every same trial;
• recovery must retain >128 lifecycle events, verify complete audit rebuild, restart and rollback recovery, and reject malformed head/truncated audit;
• SQLite storage growth is retained from the exact object's ctx.storage.sql.databaseSize, not a namespace aggregate;
• evaluator output must not echo raw samples or the caller-supplied evidence pathname;
• evaluation does not authenticate the producer, deploy the Durable Object, or substitute for issue feat(tool-capability): persist append-only external-extension lifecycle evidence #561's real remote runtime/recovery rehearsal.

Keep Draft until the unchanged exact head has application CI, reviewer-ci, Security Scan, and patch-validator-image GREEN and review remains clean.

Summary by CodeRabbit

• 새로운 기능

  • 외부 확장 수명주기 운영 증거를 검증하는 CLI 감사 기능을 추가했습니다.
  • 증거 평가 결과를 PASS/FAIL 상태와 검증 지표가 포함된 JSON으로 출력합니다.
  • 증거 파일 경로를 지정하거나 기본 경로를 사용할 수 있습니다.

• 문서

  • 운영 증거의 허용 조건, 성능 기준, 복구 검증 및 저장소 제약을 문서화했습니다.

• 테스트

  • 정상 및 실패 사례, 입력 검증, 지연 시간·동시성·복구·저장소 검증에 대한 테스트를 추가했습니다.

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: 180b66d6-d84d-451f-a4e9-64ad98ca2bd5

📥 Commits

Reviewing files that changed from the base of the PR and between 73039b5 and b7c1804.

📒 Files selected for processing (8)

• docs/doctoring/external-extension-lifecycle-operability-evidence.md
• package.json
• scripts/external-extension-lifecycle-operability-audit.mjs
• scripts/lib/external-extension-lifecycle-operability-evidence.mjs
• test/external-extension-lifecycle-operability-audit.test.ts
• test/external-extension-lifecycle-operability-evidence-edge.test.ts
• test/external-extension-lifecycle-operability-evidence.test.ts
• vitest.config.ts

---

📝 Walkthrough

Walkthrough

외부 Durable Object 운영성 증거 계약과 평가기를 추가합니다. CLI는 증거 JSON을 읽고 평가 결과를 출력합니다. 테스트는 정상 및 실패 조건을 검증하며, 관련 스크립트를 커버리지 대상에 포함합니다.

Changes

외부 확장 수명주기 운영성 증거

Layer / File(s)	Summary
증거 계약과 플랫폼 제약 docs/doctoring/external-extension-lifecycle-operability-evidence.md	원격 Cloudflare Durable Object와 SQLite만 허용하는 증거 조건을 정의합니다. 지연 샘플, p95, CAS 경쟁, 복구 감사, 저장소 증가 및 후속 단계를 기록합니다.
증거 평가 구현 scripts/lib/external-extension-lifecycle-operability-evidence.mjs	식별자, UTC 시각, 지연 샘플, nearest-rank p95, CAS 결과, 복구 증거 및 저장소 증가를 검증합니다. 모든 검사 결과와 계산 지표를 PASS 또는 FAIL로 반환합니다.
감사 CLI 연결 package.json, scripts/external-extension-lifecycle-operability-audit.mjs	operations:lifecycle-operability 명령을 추가합니다. CLI는 기본 증거 경로를 사용하고, 읽기 실패를 처리하며, 결과를 출력하고 실패 시 종료 코드 1을 설정합니다.
평가 및 CLI 검증 test/external-extension-lifecycle-operability-audit.test.ts, test/external-extension-lifecycle-operability-evidence-edge.test.ts, test/external-extension-lifecycle-operability-evidence.test.ts, vitest.config.ts	정상 증거와 입력 형태, 식별자, 샘플, p95, CAS, 복구, 저장소 및 CLI 실패 조건을 검증합니다. 두 운영성 스크립트를 커버리지 대상에 추가합니다.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant 운영자
  participant 감사CLI
  participant 증거JSON
  participant 평가기
  participant 출력
  운영자->>감사CLI: operations:lifecycle-operability 실행
  감사CLI->>증거JSON: 증거 파일 읽기
  감사CLI->>평가기: evaluateExternalExtensionLifecycleOperabilityEvidence 호출
  평가기-->>감사CLI: PASS 또는 FAIL와 계산 지표 반환
  감사CLI->>출력: JSON 결과 출력

Loading

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/lifecycle-operability-evidence

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[seonghobae]

Current-head self-review found one owned observability/privacy defect before GREEN: the audit result unnecessarily republishes the caller-supplied evidence pathname. The evaluator only needs the file as an input capability; a path can contain tenant/operator identifiers and is not acceptance evidence. Keep the output bounded to the source/check/failure/metric result and retain no raw path or raw samples.

[seonghobae]

Current-head review found one remaining evidence-integrity gap before GREEN. contended_append.latency_ms is evaluated independently from contention_trials, so structurally valid evidence can report 100 low-latency samples from an unrelated/uncontended run while proving only a handful of pairwise CAS trials. For a contended-append p95, each pairwise trial has exactly two attempts (winner + conflict loser); bind the planned latency denominator to 2 * contention_trials and cover denominator drift with a hostile regression. Do not relax the p95 or contention requirements.