multi: Add durable batch canonicality authority

batchcanon/AGENTS.md

@@ -0,0 +1,100 @@
+# batchcanon
+
+## Purpose
+
+Client-side **batch canonicality data model** for the reorg-safety epic
+(darepo#454, task C2). Holds the durable, reorg-aware record of how each batch
+(commitment) transaction is faring against the best chain: its canonicality
+state, current confirmation observation, recompute inputs for effective
+expiry, the inputs it consumes, the VTXOs it anchors, and the reverse
+dependencies needed to restore a provisionally consumed VTXO.
+
+This package is **data + query/update interface only**. It contains no
+interpretation, no chain watching, and no admission behavior — those belong to
+the (later) `BatchCanonicalityManager` and the VTXO manager. Keeping the model
+in its own package, separate from `chainsource` (raw observation) and `vtxo`
+(admission), preserves the epic's observation → interpretation → action split.
+
+## Key Types
+
+- `State` — canonicality state enum: `StateUnseen`, `StateProvisional`,
+  `StateFinalized`, `StateReorgedOut`, `StateConflictProvisional`,
+  `StateConflictFinalized`. Reorg-reversible; **no state is a terminal
+  verdict** at this layer. Persisted as an append-only typed INTEGER column —
+  values must never be renumbered.
+- `PolicyState` — reserved policy classification slot (`PolicyStateDefault`
+  only); persisted and round-tripped, no business meaning yet.
+- `Record` — per-batch record keyed by `BatchTxID`. Identity is by **txid**,
+  never `(txid, block hash)`; `ConfirmationBlock` is an observation attribute
+  only. `EffectiveExpiry()` derives the absolute expiry as
+  `ConfirmationHeight + CSVExpiryDelta`, returning `None` when unconfirmed —
+  the structural guarantee that expiry is recomputed on every
+  reconfirmation rather than frozen.
+- `ProvisionalConsumer` — reverse-dependency edge (consumed VTXO → consumer
+  batch) enabling VTXO restore if a consumer batch never becomes canonical.
+- `Availability` — derived (never persisted) VTXO-lineage spendability:
+  `AvailableFinal`, `AvailableProvisional`, `AvailabilityUnknown`,
+  `LimboReorg`, `LimboConflict`, `Invalidated`. `AvailabilityForState`
+  maps one batch's `State`; `CombineAvailability` takes the worst across a
+  multi-parent lineage; `Usable()` is true only for confirmed lineage.
+  `LineageAvailability`/`LineageBlocked` load each parent batch from the
+  `Store` and produce the combined availability / block decision the VTXO
+  manager's admission gate (C5 wiring) calls per candidate. The gate is
+  permissive: unseen / not-yet-registered lineage does not block — only
+  limbo/invalidated lineage does.
+- `Store` — behavior-free durable query/update interface. Implemented by
+  `db.BatchCanonicalityPersistenceStore` over the `000020`/`000021` schema;
+  backfilled from existing VTXOs via
+  `db.BatchCanonicalityPersistenceStore.BackfillFromVTXOs`.
+- `Manager` — the actor that interprets chain observation into canonicality
+  state (the sole client-side interpreter). Registered under
+  `ManagerServiceKey`. `RegisterBatchRequest` arms one reorg-aware
+  confirmation watch on the batch tx and one reorg-aware spend watch per
+  consumed input (deduped per batch, idempotent — repeats merge dependent
+  VTXOs). It maps chainsource `ConfirmationEvent`/`ConfReorgedEvent`/
+  `ConfDoneEvent` and `SpendEvent`/`SpendReorgedEvent`/`SpendDoneEvent` onto
+  its own mailbox and derives `State` per the priority
+  `conflict_finalized > conflict_provisional > reorged_out >
+  finalized/provisional > unseen`. `Reconcile` re-arms watches for non-final
+  batches after restart without downgrading persisted state.
+  `GetBatchStateRequest` reads the persisted record. `NewManager` returns the
+  behavior; the caller registers it, then calls `SetSelfRef(ref.TellRef())`
+  and `Reconcile`.
+
+## Relationships
+
+- **Depends on**: `btcd/chaincfg/chainhash`, `btcd/wire`, `lnd/fn/v2` only.
+- **Depended on by**: `db` (concrete store), and — in later tasks — the
+  batch canonicality manager and `vtxo` admission.
+
+## Invariants
+
+- Identity is by txid / outpoint, never by `(txid, block hash)`.
+- Expiry is never persisted as a standalone or terminal value; it is always
+  derived from `CSVExpiryDelta` + the current confirmation observation.
+- State enum integer values are append-only (persisted column).
+
+## Expiry-as-terminal audit (darepo#454 C2)
+
+C2 requires auditing every site that treats `BatchExpiry`/`Expired` as a
+one-way terminal fact. These are flagged for rework when the
+BatchCanonicalityManager (task C3/C4) rewires expiry consumers onto
+`Record.EffectiveExpiry()`; **no behavior is changed by C2**:
+
+- `vtxo/transitions.go` (`ExpiryStatusExpired → FailedState{Recoverable:
+  false}`, and the Critical/Expired escalations) — the primary offender: a
+  reorg that lowers the confirmation height could otherwise push a VTXO
+  permanently into non-recoverable `Failed`.
+- `vtxo/expiry.go` (`CheckExpiry`, `BlocksUntilExpiry`) — compute from the
+  frozen absolute `vtxo.BatchExpiry`; must consume effective (recomputable)
+  expiry instead.
+- `vtxo/actor.go` — schedules on the frozen absolute `BatchExpiry`.
+- `waved/vhtlc_recovery_target.go` — folds multiple roots into a
+  most-restrictive absolute `batchExpiry`.
+- `unroll/proof_assembler.go` (`BatchExpiry == 0`) — treats zero as "unset",
+  not terminal; benign, documented for completeness.
+
+## Deep Docs
+
+- [ARCHITECTURE.md](../ARCHITECTURE.md) — System-wide package map.
+</content>

batchcanon/CLAUDE.md

@@ -0,0 +1,100 @@
+# batchcanon
+
+## Purpose
+
+Client-side **batch canonicality authority** for the reorg-safety epic
+(lumos#454). This package is the sole client interpreter of how each batch
+(commitment) transaction is faring against the best chain, and it produces the
+**fail-closed lineage availability** the VTXO manager's admission gate and the
+round/OOR producers consume.
+
+It owns three things: the durable, reorg-aware **data model** (`Record`,
+`ConsumedInput`, `ConsumerEdge`), the dependency-light **reducer** that derives
+canonicality `State` and `Availability` from a complete current-chain
+observation, and the **`Manager`** actor that arms reorg-aware watches, drives
+the versioned snapshot/readiness restart barrier, and interprets chainsource
+observations into state.
+
+Observation (chainsource) → interpretation (this package) → admission (vtxo)
+stays a strict split: chainsource reports raw reversible facts, this package
+decides canonicality, and the VTXO manager remains the admission boundary.
+
+## Key Types
+
+- `State` — canonicality state: `StateUnseen`, `StateProvisional`,
+  `StateFinalized`, `StateReorgedOut`, `StateConflictProvisional`,
+  `StateConflictFinalized`. Every state is reorg-reversible; **no state is a
+  terminal verdict** except a conflict that reached policy finality. Priority:
+  `conflict_finalized > conflict_provisional > reorged_out >
+  finalized/provisional > unseen`. Persisted as an append-only typed INTEGER —
+  values must never be renumbered.
+- `RegistrationStage` — crash-safe evidence lifecycle: `Registering` →
+  `Reconciling` → `Complete`. Semantic `State` is **never** admissible unless
+  the stage is `Complete` and `ReadyGeneration == ObservationGeneration`.
+- `Record` — durable per-batch view keyed by **`BatchTxID`** (never
+  `(txid, block hash)`; a reorg that re-mines the same tx is the same batch).
+  Carries `BatchTx` (the serialized commitment tx, authenticated to hash to
+  `BatchTxID`), `ObservationGeneration`/`ReadyGeneration`/`Revision`,
+  `ConfirmationHeight`/`Block` (observation attributes; cleared on reorg),
+  `CSVExpiryDelta`, `ConsumedInputs`, and `DependentVTXOs`. `Ready()` is true
+  only with complete evidence + `Complete` stage + a matching ready generation.
+  `EffectiveExpiry()` derives absolute expiry on demand
+  (`ConfirmationHeight + CSVExpiryDelta`), so a reorg-and-reconfirm recomputes
+  it instead of freezing it.
+- `ConsumedInput` — one actual `TxIn` the batch spends, with `Value` +
+  `PkScript` (required to arm the reorg-aware spend watch) and persisted
+  `Conflicting`/`ConflictFinal` flags so restart reconciliation cannot
+  transiently downgrade a persisted conflict.
+- `ConsumerEdge` — the **logical value-lineage** edge (a VTXO consumed by a
+  batch), separate from the on-chain `ConsumedInputs` graph. Carries
+  `ExpectedRevision` and the full `CreatorLineage`; used by the terminal
+  conditional-restore compare-and-swap, not mislabeled as a commitment input.
+- `Availability` — derived (never persisted) VTXO-lineage spendability:
+  `AvailableFinal`, `AvailableProvisional`, `AvailabilityUnknown`,
+  `LineageReconciling`, `LimboReorg`, `LimboConflict`, `Invalidated`.
+  **Fail-closed**: a missing record, a non-`Ready()` record, or an empty
+  lineage all map to `LineageReconciling`; `Usable()` is true only for
+  `AvailableFinal`/`AvailableProvisional`. `CombineAvailability` takes the
+  worst across a multi-parent lineage.
+- `AdmissionToken` — the linearizable guard returned by a successful lineage
+  query, binding the observation generation + lineage revision; producers
+  revalidate it before each critical effect.
+- `Manager` — the actor interpreter. Arms one reorg-aware confirmation watch
+  on the batch tx plus one reorg-aware spend watch per consumed input;
+  registration cross-checks the serialized `BatchTx` (hash == `BatchTxID`,
+  output/pkScript bound, every `TxIn` registered) before a row can reach
+  `Ready`. `Reconcile(g)` runs the restart barrier: it opens a new observation
+  generation, re-arms watches, requires an explicit current fact per subject,
+  and only installs `Ready(g)` + derived state atomically — admission stays
+  closed until then, so a persisted conflict can never transiently look usable.
+
+## Relationships
+
+- **Depends on**: `btcd/chainhash`, `btcd/wire`, `lnd/fn/v2` only (plus
+  `baselib/actor` + `chainsource` for the `Manager`). The reducer/model
+  (`state.go`, `availability.go`, `record.go`) is deliberately dependency-light
+  so the **server can reuse the same reducer** (lumos#454 Server PR2).
+- **Depended on by**: `db` (concrete `Store`), `vtxo` (admission gate), and the
+  round/OOR producers (registration before exposure).
+
+## Invariants
+
+- Identity is by txid / outpoint, never `(txid, block hash)`.
+- Fail-closed: missing / incomplete / unarmed / reconciling lineage is never
+  usable. Registration completeness is part of the safety proof, not a
+  compatibility hint.
+- Registration authenticates the serialized commitment tx (hash + full `TxIn`
+  set) before a record reaches `Ready`; an omitted or unauthenticated input
+  keeps the record unavailable.
+- Two graphs: the on-chain `ConsumedInputs` (`TxIn`) graph drives conflict
+  observation; the logical `ConsumerEdge` graph drives inherited lineage and
+  conditional restore.
+- Expiry is never persisted as a standalone/terminal value; always derived.
+- State enum integer values are append-only (persisted column).
+- Restart increments the observation generation before any watch is armed;
+  admission is closed until `Ready(g)` installs a complete snapshot.
+
+## Deep Docs
+
+- [ARCHITECTURE.md](../ARCHITECTURE.md) — System-wide package map.
+- `REORG_SAFETY_SPEC.md` (workspace root) — normative §3–§9 contracts.

batchcanon/admission_test.go

@@ -0,0 +1,154 @@
+package batchcanon
+
+import (
+	"errors"
+	"testing"
+
+	"github.com/btcsuite/btcd/chainhash/v2"
+	"github.com/stretchr/testify/require"
+)
+
+// TestAdmissionTokenTracksReadyGenerationAndRevision proves that admission
+// remains closed until every watched subject contributes to Ready(g), and that
+// a later canonicality change invalidates the issued token before a critical
+// side effect can use it.
+func TestAdmissionTokenTracksReadyGenerationAndRevision(t *testing.T) {
+	t.Parallel()
+
+	h := newManagerHarness(t, 100)
+	txid := testBatchTxid(0xa7)
+	input := testOutpoint(0xa8, 0)
+
+	h.registerBatch(t, &RegisterBatchRequest{
+		BatchTxID:            txid,
+		ConfirmationPkScript: []byte{0x51, 0x20, 0xa7},
+		ConsumedInputs:       []ConsumedInput{ci(input)},
+	})
+
+	query := func() *QueryLineageResponse {
+		resp, err := h.mgrRef.Ask(
+			t.Context(), &QueryLineageRequest{
+				BatchTxIDs: []chainhash.Hash{txid},
+			},
+		).Await(t.Context()).Unpack()
+		require.NoError(t, err)
+		lineage, ok := resp.(*QueryLineageResponse)
+		require.True(t, ok)
+
+		return lineage
+	}
+
+	// Merely arming every watch is not Ready(g): no subject has supplied a
+	// current observation yet.
+	result := query()
+	require.Equal(t, LineageReconciling, result.Availability)
+	require.Nil(t, result.Token)
+
+	// The confirmation alone is still incomplete because the actual input
+	// spend has not been observed for this generation.
+	h.fireConfirmed(t, txid, 101, testBatchTxid(0xb1))
+	result = query()
+	require.Equal(t, LineageReconciling, result.Availability)
+	require.Nil(t, result.Token)
+
+	// The batch's own spend supplies the final subject observation.
+	// Ready(g) is installed and a revision-bound token can now be issued.
+	h.fireSpend(t, input, txid, 101)
+	result = query()
+	require.Equal(t, AvailableProvisional, result.Availability)
+	require.NotNil(t, result.Token)
+	require.Len(t, result.Token.Lineage, 1)
+	token := *result.Token
+
+	validation, err := h.mgrRef.Ask(
+		t.Context(), &ValidateAdmissionRequest{Token: token},
+	).Await(t.Context()).Unpack()
+	require.NoError(t, err)
+	valid, ok := validation.(*ValidateAdmissionResponse)
+	require.True(t, ok)
+	require.True(t, valid.Valid)
+	require.Equal(t, AvailableProvisional, valid.Availability)
+
+	// A reorg changes semantic availability and the durable revision. The
+	// old token is stale and cannot cross a point of no return.
+	h.fireConfReorged(t, txid)
+	validation, err = h.mgrRef.Ask(
+		t.Context(), &ValidateAdmissionRequest{Token: token},
+	).Await(t.Context()).Unpack()
+	require.NoError(t, err)
+	valid, ok = validation.(*ValidateAdmissionResponse)
+	require.True(t, ok)
+	require.False(t, valid.Valid)
+	require.Equal(t, LimboReorg, valid.Availability)
+}
+
+// TestAdmissionFailsClosedOnObservationPersistenceError proves the manager's
+// in-memory overlay cannot issue or validate a token from an old durable
+// usable row after a newer chain observation failed to commit.
+func TestAdmissionFailsClosedOnObservationPersistenceError(t *testing.T) {
+	t.Parallel()
+
+	h := newManagerHarness(t, 100)
+	txid := testBatchTxid(0xb7)
+	input := testOutpoint(0xb8, 0)
+	h.registerBatch(t, &RegisterBatchRequest{
+		BatchTxID:            txid,
+		ConfirmationPkScript: []byte{0x51, 0x20, 0xb7},
+		ConsumedInputs:       []ConsumedInput{ci(input)},
+	})
+	h.fireConfirmed(t, txid, 101, testBatchTxid(0xc1))
+	h.fireSpend(t, input, txid, 101)
+
+	query := func() *QueryLineageResponse {
+		resp, err := h.mgrRef.Ask(
+			t.Context(), &QueryLineageRequest{
+				BatchTxIDs: []chainhash.Hash{txid},
+			},
+		).Await(t.Context()).Unpack()
+		require.NoError(t, err)
+		queryResp, ok := resp.(*QueryLineageResponse)
+		require.True(t, ok)
+
+		return queryResp
+	}
+
+	admitted := query()
+	require.Equal(t, AvailableProvisional, admitted.Availability)
+	require.NotNil(t, admitted.Token)
+	oldToken := *admitted.Token
+
+	h.store.setApplyError(errors.New("injected durable write failure"))
+	h.fireConfReorged(t, txid)
+
+	// The SQL-equivalent fake still contains the previously usable row, but
+	// the serialized manager observed a newer event and must fail closed.
+	durable, err := h.store.GetBatch(t.Context(), txid)
+	require.NoError(t, err)
+	require.Equal(t, StateProvisional, durable.State)
+	require.True(t, durable.Ready())
+	blocked := query()
+	require.Equal(t, LineageReconciling, blocked.Availability)
+	require.Nil(t, blocked.Token)
+
+	resp, err := h.mgrRef.Ask(
+		t.Context(), &ValidateAdmissionRequest{Token: oldToken},
+	).Await(t.Context()).Unpack()
+	require.NoError(t, err)
+	validation, ok := resp.(*ValidateAdmissionResponse)
+	require.True(t, ok)
+	require.False(t, validation.Valid)
+	require.Equal(t, LineageReconciling, validation.Availability)
+
+	// A later full-snapshot write can safely recover without replaying the
+	// failed operation. It persists all retained in-memory observations and
+	// issues a different revision-bound token.
+	h.store.setApplyError(nil)
+	h.fireConfirmed(t, txid, 102, testBatchTxid(0xc2))
+	recovered := query()
+	require.Equal(t, AvailableProvisional, recovered.Availability)
+	require.NotNil(t, recovered.Token)
+	require.NotEqual(
+		t, oldToken.Lineage[0].Revision,
+		recovered.Token.Lineage[0].Revision,
+	)
+}