Skip to content

darepod: activate the batch-canonicality reorg-safety gate (C9) - #822

Closed
ellemouton wants to merge 2 commits into
c8-unroll-source-lineage-gatefrom
c9-darepod-canonicality-activation
Closed

darepod: activate the batch-canonicality reorg-safety gate (C9)#822
ellemouton wants to merge 2 commits into
c8-unroll-source-lineage-gatefrom
c9-darepod-canonicality-activation

Conversation

@ellemouton

@ellemouton ellemouton commented Jun 29, 2026

Copy link
Copy Markdown
Member

What

Stacked on #C8 (c8-unroll-source-lineage-gate). This is the activation flag day for the batch-canonicality reorg-safety gate (epic #454), the first two F-series acceptance tests proving the live gate blocks, and a real-bug fix the F3 test surfaced. Slices C5–C8 landed every gate field defaulting to nil/None (dormant no-op); this PR wires it live in darepod and proves the whole pipeline against real bitcoind reorgs.

Commits

  1. systest: Prove batch canonicality survives a real reorg — manager re-anchors a batch across a real reorg.
  2. oor: Forward the canonicality manager ref to session actorsOORRegistryConfig field + childConfig propagation.
  3. darepod: Activate the batch-canonicality reorg-safety gate — build store, backfill, register+reconcile manager, thread store into VTXO+unroll gates and the manager ref into round+OOR.
  4. darepod: Document the batch-canonicality activation wiring.
  5. harness: Add ReorgExcludingMempool for off-chain reorg windows — mines empty replacement blocks so a tx stays off-chain in a stable ReorgedOut window.
  6. systest: Prove the gate blocks a reorged-out VTXO (F2) — VTXO excluded from coin selection while its batch is reorged out, admitted on reconfirm.
  7. db: Persist the pkScript of each batch consumed inputinput_pk_script column + sqlc.
  8. multi: Thread consumed-input pkScripts to the spend watchthe bug fix (see below).
  9. harness: Spend a chosen outpoint and expose its pkScript.
  10. systest: Prove the gate blocks an input-conflicted VTXO (F3) — VTXO excluded on ConflictProvisional, admitted when the conflicting spend is reorged away.
  11. systest: Prove the gate blocks a reorged-out ancestor (F4) — a VTXO whose ancestor batch (not its direct commitment) reorgs out is excluded, admitted on ancestor reconfirm; proves the multi-parent lineage-depth dimension.
  12. darepod: Make the reorg-safety depth operator-configurable--reorgsafetydepth with a network-aware default (6; 100 on testnet, whose difficulty-reset rule produces deep reorgs). Bounds the deepest reorg the daemon detects.
  13. batchcanon: Restore forfeited VTXOs when their batch is invalidatedF6 manager half: records reverse-dependency edges (ForfeitedVTXOs) and, on the consumer batch reaching ConflictFinalized, restores them via a RestoreConsumedVTXO callback (drops edges on Finalized).
  14. multi: Add a VTXO-manager restore path for forfeited VTXOsRestoreForfeitedVTXORequest + a handler that re-materializes a forfeited VTXO to Live from its descriptor (reuses the unilateral-exit recovery pattern; no FSM-lifecycle change).
  15. darepod: Wire forfeit restore to the canonicality manager — the restore callback → VTXO manager.
  16. round: Declare forfeited VTXOs to the canonicality managerVTXOCreatedNotification.ForfeitedVTXOsRegisterBatchRequest.
  17. systest: Prove a forfeited VTXO is restored on invalidation (F6) — round-2 commitment invalidated → round-1 VTXO restored to Live + selectable.

Bug found + fixed by F3

Writing F3 surfaced a real production bug: batchcanon.armSpendWatch registered the per-input spend watch with no pkScript, so lnd rejected it ("an output script must be provided") and conflict detection (LimboConflict) never armed against a real backend. Round-born batches register consumed inputs, so this silently disabled the double-spend half of the reorg-safety gate in production (confirmation-based reorg tracking was unaffected). Commits 7–9 thread the consumed input's pkScript end to end (manager → RegisterBatchRequest/Record → DB → reconcile, sourced in round from the commitment PSBT's witness UTXOs); an input with no script is skipped (logged) rather than failing the whole batch registration.

Proof (real bitcoind+LND systests, all green)

  • TestBatchCanonicalityReorgRoundTrip — re-anchors across a real reorg.
  • TestBatchCanonicalityGateBlocksReorgedVTXO (F2) — SelectAndReserve fails while reorged out, succeeds on reconfirm.
  • TestBatchCanonicalityGateBlocksConflictedVTXO (F3) — SelectAndReserve fails on ConflictProvisional, succeeds when the conflict is reorged away.
  • TestBatchCanonicalityGateBlocksReorgedAncestor (F4) — ancestor reorged out blocks the VTXO while its direct commitment stays Provisional; admitted on ancestor reconfirm.
  • TestBatchCanonicalityRestoresForfeitedVTXO (F6) — a round-2 commitment driven to ConflictFinalized restores the round-1 VTXO it forfeited back to Live + selectable.
  • TestSendVTXOEndToEnd — full daemon boots + happy path unaffected with the gate live.
  • go build ./..., go vet, make lint-changed-local (0 issues), make commitmsg-lint, batchcanon/db/round/darepod/oor unit tests green.

F-series status

F2 (reorg), F3 (input-conflict), F4 (ancestor-depth), and F6 (forfeit-reversal restore) are all proven by systest. F1 (boarding-input conflict ≡ F3) and F5 (ancestor-input conflict ≡ F3+F4) are covered by code-path equivalence. The reorg-safety gate is live and proven on every failure mode it models.

Note on the F6 restore design

F6 restores a forfeited VTXO when its consumer batch is invalidated. Rather than introduce a new non-terminal "provisionally forfeited" FSM state (which would have made every forfeit provisional and required finalize-wiring for the common refresh/leave path), the forfeit still goes terminal but is reversible: the manager re-materializes the VTXO to Live from its persisted descriptor on invalidation, reusing the unilateral-exit recovery pattern. This is lower-risk and avoids regressing the common path; the trade-off is the VTXO is briefly terminal-then-resurrected rather than staying provisional throughout the reorg window.

Part of #454.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request integrates a new batch-canonicality subsystem (including a durable store and manager actor) to provide reorg-safety gates for VTXO coin-selection, unroll-admission, round, and OOR paths. It also adds a comprehensive system test to verify the end-to-end reorg pipeline. The review feedback highlights a critical concurrency hazard where mgr.Reconcile is called directly on the manager behavior struct after registering it as an actor, which should be resolved by using actor messaging. Additionally, minor improvements are suggested in the new system test to prevent potential panics from out-of-bounds slice access and nil pointer dereferences.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

Comment thread darepod/server.go
Comment on lines +4069 to +4074
if err := mgr.Reconcile(ctx); err != nil {
s.actorSystem.StopAndRemoveActor("batch-canonicality")

return fmt.Errorf("unable to reconcile batch canonicality: %w",
err)
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

Data Race / Concurrency Hazard: Calling mgr.Reconcile(ctx) directly on the mgr behavior struct from the main thread after registering it with the actor system (actor.RegisterWithSystem) introduces a data race on the internal m.watches map.

Once RegisterWithSystem is called, the actor's event loop is started on a separate goroutine. Any incoming messages (such as RegisterBatchRequest from other actors or chainsource events) will execute Receive and access/mutate m.watches concurrently with the main thread executing Reconcile.

Suggested Solution

Define a ReconcileRequest message in the batchcanon package (implementing ManagerMsg) and handle it within Manager.Receive. Then, perform the reconciliation by sending an Ask to the registered actor reference:

	if _, err := mgrRef.Ask(ctx, &batchcanon.ReconcileRequest{}).Await(ctx).Unpack(); err != nil {
		s.actorSystem.StopAndRemoveActor("batch-canonicality")
		return fmt.Errorf("unable to reconcile batch canonicality: %w", err)
	}

This guarantees that reconciliation runs entirely within the actor's single-threaded execution context, eliminating any concurrency issues.

Suggested change
if err := mgr.Reconcile(ctx); err != nil {
s.actorSystem.StopAndRemoveActor("batch-canonicality")
return fmt.Errorf("unable to reconcile batch canonicality: %w",
err)
}
if _, err := mgrRef.Ask(ctx, &batchcanon.ReconcileRequest{}).Await(ctx).Unpack(); err != nil {
s.actorSystem.StopAndRemoveActor("batch-canonicality")
return fmt.Errorf("unable to reconcile batch canonicality: %w",
err)
}

Comment on lines +151 to +154
require.Equal(
t, originalBlocks[0].Hash, reorg.Disconnected[0].Hash,
"the reorg should have disconnected the confirmation block",
)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Potential Panic (Index Out of Range): Accessing reorg.Disconnected[0] directly without first asserting that reorg.Disconnected is not empty can cause a panic if the reorg fails or behaves unexpectedly.

Asserting the length of reorg.Disconnected first ensures a clean test failure instead of a panic.

Suggested change
require.Equal(
t, originalBlocks[0].Hash, reorg.Disconnected[0].Hash,
"the reorg should have disconnected the confirmation block",
)
require.Len(t, reorg.Disconnected, 1, "expected exactly one disconnected block")
require.Equal(
t, originalBlocks[0].Hash, reorg.Disconnected[0].Hash,
"the reorg should have disconnected the confirmation block",
)

Comment on lines +255 to +258
got, ok := resp.(*batchcanon.GetBatchStateResponse)
if !ok || !got.Found {
return false
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Defensive Programming / Potential Nil Dereference: If got.Found is true but got.Record is nil (due to an unexpected issue or mock behavior), calling pred(got.Record) will panic because pred dereferences the record (e.g., rec.State on line 219).

Adding a nil check for got.Record before calling pred prevents potential panics.

Suggested change
got, ok := resp.(*batchcanon.GetBatchStateResponse)
if !ok || !got.Found {
return false
}
got, ok := resp.(*batchcanon.GetBatchStateResponse)
if !ok || !got.Found || got.Record == nil {
return false
}

@ellemouton
ellemouton force-pushed the c9-darepod-canonicality-activation branch 2 times, most recently from 2000ae8 to 0306211 Compare June 30, 2026 17:32
@ellemouton
ellemouton force-pushed the c8-unroll-source-lineage-gate branch from e2ecc19 to f5ab5f3 Compare July 1, 2026 16:16
@ellemouton
ellemouton force-pushed the c9-darepod-canonicality-activation branch from dafebae to 54c1281 Compare July 1, 2026 16:16
@levmi levmi added daemon P0 Priority 0 — blocker reorg safety Fund-safety: stuck, lost, or mis-counted funds vtxo labels Jul 6, 2026
@litbot-9000

Copy link
Copy Markdown
Collaborator

@ellemouton, remember to re-request review from reviewers when ready

Squashed for the btcd v2 port. Unroll gates fresh admission on the
source VTXO's batch-lineage canonicality (blocks only Invalidated,
fail-permissive).
@ellemouton
ellemouton force-pushed the c8-unroll-source-lineage-gate branch from f5ab5f3 to 7e56139 Compare July 8, 2026 21:13
@ellemouton
ellemouton force-pushed the c9-darepod-canonicality-activation branch from 54c1281 to 94a41fe Compare July 8, 2026 21:19
@ellemouton
ellemouton force-pushed the c8-unroll-source-lineage-gate branch from 7e56139 to 4bf4885 Compare July 8, 2026 22:43
@ellemouton
ellemouton force-pushed the c9-darepod-canonicality-activation branch from 94a41fe to 585d03a Compare July 8, 2026 22:43
Squashed for the btcd v2 port. Flag-day activation: darepod builds the
batchcanon store, backfills from VTXOs at best height, registers +
reconciles the manager, and threads the store into vtxo/unroll configs
and fn.Some(ref) into round/oor. Includes the F-series reorg systests
(F2/F3/F4/F6, ReorgExcludingMempool harness helper) + reorg-safety
depth config and the consumed-input pkScript fix.
@ellemouton

Copy link
Copy Markdown
Member Author

Superseded by #896 as part of condensing the reorg-safety client stack (epic lightninglabs/darepo#454) from 12 PRs into 3. The commits are carried over unchanged; see #896. Branch retained as a backup.

@ellemouton ellemouton closed this Jul 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

daemon P0 Priority 0 — blocker reorg safety Fund-safety: stuck, lost, or mis-counted funds vtxo

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants