feat: tamper-evident receipt chain with signed checkpoints (#758) - #770
Merged
Conversation
…sh-chain # Conflicts: # docs/CHANGELOG.md # docs/ai-act-transparency.md # docs/middleware-agent-handoff.md # docs/security-architecture.md # middleware/.env.example # middleware/src/config.ts # middleware/src/index.ts # middleware/src/receipts/store.ts # middleware/test/turnReceipts.test.ts
…sh-chain # Conflicts: # docs/CHANGELOG.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #758. Builds directly on the
turn_receiptstable from #757 (PR #763, merged) — the branch is fully synced withmain(incl. #759/#760).What
The receipt record (#757) becomes tamper-evident: every row joins a hash chain, appends are serialized into one linear chain, and periodic Ed25519 checkpoints — signed with a key the database never holds — anchor the head outside the DB. Until now a DB admin could edit any audit row without detection; now an edit breaks the chain visibly at the exact row, and a wholesale rewrite fails against the signed checkpoints.
Threat model (stated, not implied): detection, not prevention. Destruction shows as sequence gaps and orphaned checkpoints. Per-row time is anchored by checkpoint cadence, not per-row (
created_atstays outside the hash — a DB default cannot be hashed before it exists).How
0041(0039 = Persist per-turn audit receipts (receipt store) #757, 0040 = Privacy Shield: operator deny-lists, miss-report review queue, idnum coverage, eval CI gate #760/PR feat: privacy shield operator deny-lists, miss queue, idnum, eval gate (#760) #766): nullable chain columns onturn_receipts(NULL = documented "pre-chain era"),audit_stream_heads(theFOR UPDATEserialization point — concurrent turns form one linear chain, no forks),audit_checkpoints, and aBEFORE UPDATEtrigger — UPDATE is never legitimate on this table; DELETE stays legal for bounded retention, and deletions are detectable as gaps.chain.ts(pure, the Provenance verification surface: verify endpoint, signed export, offline verifier, chain view #761 foundation):canonicalJson(JCS subset — sorted keys, JSON.stringify framing), newline-framedcomputeEntryHash= sha256(stream ‖ seq ‖ hex(prev) ‖ canonical(payload)),verifyChainSegmentdistinguishinghash_mismatch/link_mismatch/seq_gap,hash_versionfor future format evolution.store.ts): BEGIN → lock head → compute → INSERT (idempotent on turn_id) → advance head → COMMIT. A replayed turn ROLLS BACK the whole transaction — no phantom head movement (test-asserted). The hash payload is the exportedreceiptChainPayloadshape so the Provenance verification surface: verify endpoint, signed export, offline verifier, chain view #761 verifier recomputes byte-identically.checkpoints.ts): key fromAUDIT_SIGNING_KEY(base64 PKCS#8; keygen script included) — env/secret-manager ONLY, never Postgres, or the admin the chain defends against could re-sign a rewritten chain. Interval worker (default 60 min), skip-when-head-unmoved, optional external JSONL anchor (AUDIT_ANCHOR_PATH) for WORM storage, documented signing-input byte layout, public key + fingerprint atGET /api/v1/operator/provenance/public-key. No key configured ⇒ loud boot warning, never a silent no-op.Tests (10 new + adapted #757 suite)
Canonicalization (key-order independence), chain build + verify, three tamper kinds (edit a payload →
hash_mismatchat exactly that seq; delete a row →seq_gap; forge a suffix →link_mismatchagainst the trusted genesis), the transactional append against a stateful fake pool producing a chain thatverifyChainSegmentaccepts end-to-end, replay-rollback invariants (no row, no head, no counter), signer rejection of RSA/garbage keys, sign→verify roundtrip with the exported public key, external anchor line round-trip, checkpoint dedup, empty-stream no-op.dist/): storingprev_hashwhereentry_hashbelongs turns the chain-verification test red.typecheck+typecheck:testgreen.Docs
CHANGELOG, handoff,
docs/security-architecture.md§7b, anddocs/ai-act-transparency.md— updated to say the mechanism now exists while keeping "cryptographically verifiable" a public non-claim until #761 ships the verify surface (a proof only we can run is not yet a proof).Pre-PR review round (self-review findings, all addressed)
FOR UPDATEon a nonexistent head row locks nothing, so two concurrent FIRST appends on a fresh deployment both computed seq 1 and the loser's receipt was permanently lost. Fixed structurally: migration0041seeds the head row (seq 0, the hard-coded sha256 genesis literal, reproduction command in the header) so the lock always has a row to grab; test asserts the seeded state yields byte-identical appends to the fallback path.rowCount === 1); test added.receiptChainPayloadJSON-round-trips before hashing, so hash input and stored JSONB see the identical plain-JSON value (toJSON-bearing values would otherwise cause guaranteed spurious mismatches in the Provenance verification surface: verify endpoint, signed export, offline verifier, chain view #761 verifier)./provenance/public-keyis always-on (configured:falseon keyless deployments instead of a 404); handoff doc-pointer corrected.Out (deliberate, per issue)
admin_auditchaining (phase 2), RFC 3161 / S3-Object-Lock anchor sinks (the JSONL anchor is the seam), and the whole verification surface (#761: verify endpoint, signed export, zero-dependency offline verifier, chain-status UI).Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.