diff --git a/CHANGELOG.md b/CHANGELOG.md index 41b5cc52b..ca6d39ec3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,6 +3,8 @@ - Add a Noema-owned exact-claim evidence receipt contract whose execution and research producers serialize one canonical artifact that binds every receipt semantic field, including command/result/isolation/network or source revision/excerpt/retrieval policy. Admission accepts only a receipt ID from untrusted model output. The owner API first verifies the exact authenticated OpenCode-handoff manifest digest, canonical envelope bytes, reviewed producer-to-kind policy, and repository/head/workflow/run/attempt identity before it can construct an immutable typed index; admission then reconstructs each canonical artifact and verifies time/claim/artifact identity. The version-2 manifest now binds a separate producer-authenticated `ClaimEvidenceRequirement` containing the exact claim, independently required evidence kind, and `context` or `finding` publication authority. Raw current-head source lines are context only: they are withheld from finding-reference prompts and cannot publish a finding or `request_changes`; an explicitly producer-authorized source finding remains usable and retains exact path/line checks. Finding-free model `request_changes` and `blocked` verdicts cannot bypass receipt admission to publish a vacuous blocking review. Requirement/receipt kind mismatch, fixed-artifact semantic substitution, caller-supplied receipt dictionaries, model self-classification, stale identities, cross-kind receipts, marker-only sandbox output, noncanonical artifact bytes, and expired receipts fail closed before the GitHub publisher. This remains the owner prerequisite for ContextualWisdomLab/.github#1641 and issue #555. The reviewed `sandboxed_verify` adapter exists in owner source, but its actual central stdout/stderr/marker-to-manifest wiring and the trusted research producer are not yet integrated; exact-head hosted GREEN, immutable release, and the verified central consumer bump remain required. ## Unreleased +- Agent Runtime의 procedural candidate screening decision을 process-local provenance로 제한한다. `assessProceduralCandidate()`가 기존 lineage·held-out·safety·score 검증을 마친 결과만 locally admitted decision으로 발행하고, State / Checkpoint나 Policy / Approval 경계가 구조만 복사·직렬화·프록시·직접 생성한 lookalike를 `assertProceduralCandidateDecision()`으로 실패-폐쇄한다. 이 brand는 evaluator authentication·persistence·approval·activation authority가 아니며 모든 decision의 `activationAuthorized:false`는 유지된다. issue #584, PR #591. +- Agent Runtime의 screened procedural decision에 baseline/candidate paired evaluation receipt의 canonical SHA-256 identity를 결합한다. exact graph/context/case-set/score/safety validation 뒤 admitted holdout 순서로 정규화한 증거 semantics를 해시하므로 observation 배열 순서만 바꿔 identity를 우회할 수 없고 실제 score/safety evidence가 바뀌면 identity도 바뀐다. 이 digest는 향후 authenticated evaluator evidence와 durable State / Checkpoint retention을 정확히 결합하기 위한 local evidence identity일 뿐 signature·producer authentication·Policy / Approval·persistence·activation authority를 부여하지 않는다. issue #584, PR #592. - Agent Runtime의 workflow-backed procedural guidance가 매 판단마다 기존 execution-scoped `NOEMA_WORKFLOW_STATE`의 current Workflow / Task Execution evidence를 다시 읽는다. re-admitted plan과 locally admitted procedural session의 canonical execution identity가 다르면 Durable Object를 선택하거나 읽기 전에 실패-폐쇄하고, 현재 cancellation·terminal·pre-start evidence는 guidance를 억제한다. 이 ACL은 두 번째 lifecycle DB, task/lifecycle mutation, retry, tool, Policy / Approval, provider routing 또는 activation authority를 만들지 않으며 non-workflow lifecycle freshness와 deployed Durable Object compatibility/p95/recovery는 별도 acceptance로 남긴다. issue #584, ADR 0017. - Agent Runtime에 tenant/task/execution-scoped immutable procedural graph와 bounded advisory context, paired held-out candidate screening을 추가한다. 모든 candidate decision은 `activationAuthorized: false`를 유지하고 tool·retry·Policy/Approval·provider routing·credential·foreign-domain authority를 부여하지 않는다. 그래프/평가 wire contract는 아직 Noema-local이며 cross-service publication은 context-graph-contracts의 immutable release를 기다린다. issue #584, ADR 0017. - Agent Runtime의 procedural guidance를 locally admitted session brand와 canonical execution lifecycle에 결합한다. 구조만 흉내 낸 session은 callback/property를 읽기 전에 거부하고, guidance는 동일 execution의 `running` 상태에서만 반환하며 accepted·cancellation-requested·terminal 상태에서는 context request를 읽지 않고 억제한다. 결과는 계속 `advisory_only`이고 tool·retry·Policy/Approval·transition authority를 만들지 않는다. issue #584. diff --git a/src/agent-runtime/procedural-evolution.ts b/src/agent-runtime/procedural-evolution.ts index 2dc1ed688..8f6f44b48 100644 --- a/src/agent-runtime/procedural-evolution.ts +++ b/src/agent-runtime/procedural-evolution.ts @@ -12,6 +12,8 @@ export interface ProceduralCandidateDecision { readonly baselineDigest: string; readonly candidateDigest: string; readonly contextDigest: string; + readonly baselineReceiptDigest: string; + readonly candidateReceiptDigest: string; readonly rejectionKey: string; readonly baselineMean: number; readonly candidateMean: number; @@ -61,11 +63,27 @@ function observations(input: unknown, graphDigest: string, contextDigest: string return holdout.map(id => byId.get(id)!); } +async function evaluationReceiptDigest( + graphDigest: string, + contextDigest: string, + holdout: readonly string[], + rows: readonly Observation[], +): Promise { + return proceduralHash([ + "noema.procedural-evaluation-receipt/v1", + graphDigest, + contextDigest, + holdout.map((caseId, index) => [caseId, rows[index].score, rows[index].safetyViolations]), + ]); +} + /** * Screens supplied paired held-out evidence for a direct child graph while keeping activation and * publication outside this pure port. The validator owner must authenticate receipts and pre-register * the evaluation context; this function checks exact identities, complete paired cases, leakage, * finite normalized scores, reported safety violations, mean non-regression, and contextual rejection. + * Returned receipt digests bind the exact validated paired evidence semantics for later authentication + * and durable retention, but a digest alone does not authenticate its producer or authorize approval. * @param input Exact-key baseline, candidate, evaluation plan, paired receipts, and prior rejection keys. * @returns Promise resolving to a frozen non-authoritative screening decision with activation always false. */ @@ -84,6 +102,8 @@ export async function assessProceduralCandidate(input: unknown): Promise training.has(id))) rejectProceduralInput("holdout_leakage"); const oldRows = observations(value.baselineReceipt, baseline.digest, contextDigest, holdout); const newRows = observations(value.candidateReceipt, candidate.digest, contextDigest, holdout); + const baselineReceiptDigest = await evaluationReceiptDigest(baseline.digest, contextDigest, holdout, oldRows); + const candidateReceiptDigest = await evaluationReceiptDigest(candidate.digest, contextDigest, holdout, newRows); const rejectedKeys = new Set(readProceduralArray(value.rejectedKeys, 0, 10_000).map(proceduralDigest)); const baselineMean = oldRows.reduce((total, row) => total + row.score, 0) / holdout.length; const candidateMean = newRows.reduce((total, row) => total + row.score, 0) / holdout.length; @@ -94,7 +114,7 @@ export async function assessProceduralCandidate(input: unknown): Promise row.safetyViolations > 0)) reason = "safety_violation"; else if (candidateMean < baselineMean) reason = "score_regression"; else reason = "validation_non_regression"; - const decision = Object.freeze({eligibleForApproval: reason === "validation_non_regression", activationAuthorized: false as const, reason, baselineDigest: baseline.digest, candidateDigest: candidate.digest, contextDigest, rejectionKey, baselineMean, candidateMean}); + const decision = Object.freeze({eligibleForApproval: reason === "validation_non_regression", activationAuthorized: false as const, reason, baselineDigest: baseline.digest, candidateDigest: candidate.digest, contextDigest, baselineReceiptDigest, candidateReceiptDigest, rejectionKey, baselineMean, candidateMean}); admittedCandidateDecisions.add(decision); return decision; } catch (error) { return normalizeProceduralError(error); } diff --git a/test/procedural-evidence-identity.test.mjs b/test/procedural-evidence-identity.test.mjs new file mode 100644 index 000000000..dbd4da6b0 --- /dev/null +++ b/test/procedural-evidence-identity.test.mjs @@ -0,0 +1,87 @@ +import { test } from "vitest"; +import assert from "node:assert/strict"; +import { createProceduralGraph } from "../src/agent-runtime/procedural-graph.ts"; +import { assessProceduralCandidate } from "../src/agent-runtime/procedural-evolution.ts"; + +const raw = () => ({ + schemaVersion: "noema.procedural-graph/v1", + tenantId: "tenant-a", + taskType: "repair", + graphId: "graph-a", + revision: 1, + parentDigest: null, + nodes: ["Start", "check"], + edges: [{ + from: "Start", + relation: "requires", + to: "check", + condition: "", + guidance: "Check evidence", + pitfalls: "", + }], +}); + +async function fixture() { + const baseline = await createProceduralGraph(raw()); + const candidateInput = raw(); + candidateInput.revision = 2; + candidateInput.parentDigest = baseline.digest; + candidateInput.edges[0].guidance = "Check exact-head evidence"; + const candidate = await createProceduralGraph(candidateInput); + const contextDigest = "c".repeat(64); + const plan = { + contextDigest, + minimumCases: 2, + trainingCaseIds: ["train-1"], + holdoutCaseIds: ["case-1", "case-2"], + }; + const receipt = (graph, firstScore, secondScore) => ({ + graphDigest: graph.digest, + contextDigest, + observations: [ + { caseId: "case-1", score: firstScore, safetyViolations: 0 }, + { caseId: "case-2", score: secondScore, safetyViolations: 0 }, + ], + }); + return { + baseline, + candidate, + plan, + baselineReceipt: receipt(baseline, 0.5, 0.6), + candidateReceipt: receipt(candidate, 0.7, 0.8), + rejectedKeys: [], + }; +} + +test("screening decision binds canonical identities of both paired evaluation receipts", async () => { + const data = await fixture(); + const first = await assessProceduralCandidate(data); + assert.match(first.baselineReceiptDigest, /^[0-9a-f]{64}$/); + assert.match(first.candidateReceiptDigest, /^[0-9a-f]{64}$/); + assert.notEqual(first.baselineReceiptDigest, first.candidateReceiptDigest); + + data.baselineReceipt.observations.reverse(); + data.candidateReceipt.observations.reverse(); + const reordered = await assessProceduralCandidate(data); + assert.equal(reordered.baselineReceiptDigest, first.baselineReceiptDigest); + assert.equal(reordered.candidateReceiptDigest, first.candidateReceiptDigest); +}); + +test("receipt identity changes when validated score evidence changes", async () => { + const data = await fixture(); + const first = await assessProceduralCandidate(data); + data.candidateReceipt.observations[0].score = 0.71; + const changed = await assessProceduralCandidate(data); + assert.notEqual(changed.candidateReceiptDigest, first.candidateReceiptDigest); + assert.equal(changed.baselineReceiptDigest, first.baselineReceiptDigest); +}); + +test("receipt identity changes when validated safety evidence changes", async () => { + const data = await fixture(); + const first = await assessProceduralCandidate(data); + data.candidateReceipt.observations[0].safetyViolations = 1; + const changed = await assessProceduralCandidate(data); + assert.equal(changed.reason, "safety_violation"); + assert.notEqual(changed.candidateReceiptDigest, first.candidateReceiptDigest); + assert.equal(changed.baselineReceiptDigest, first.baselineReceiptDigest); +});