docs: nightly doc-gardening sweep 2026-06-08

ARCHITECTURE.md

@@ -50,6 +50,7 @@ package may import from a higher layer.
 | [`db`](db/) | SQLite/PostgreSQL persistence: boarding, rounds, VTXOs, OOR artifacts, fee ledger |
 | [`mailbox`](mailbox/) | Mailbox protocol primitives across three sub-packages (pb, rpc, conn) |
 | [`serverconn`](serverconn/) | Unified server connector: durable egress, ingress polling, unary RPC facade |
+| [`serverconn/mailboxpull`](serverconn/mailboxpull/) | Shared exponential-backoff retry primitives for mailbox pull loops (used by serverconn ingress and SDK swap consumers) |
 
 ### Layer 3: Application & Orchestration
 
@@ -77,6 +78,8 @@ package may import from a higher layer.
 
 | Package | Purpose |
 |---------|---------|
+| [`p-models`](p-models/) | Executable P formal models and Go conformance bridge for distributed-systems properties (durable mailbox, Read/Commit fence) |
+| [`p-models/durableactor/bridge`](p-models/durableactor/bridge/) | Go conformance harness: replays P model mailbox traces against the real `db/actordelivery` store |
 | [`harness`](harness/) | Docker-based Bitcoin/LND integration test environment |
 | [`systest`](systest/) | System-level end-to-end tests |
 | [`internal/actortest`](internal/actortest/) | Durable actor integration tests with real DB backends |

baselib/actor/AGENTS.md

@@ -28,7 +28,8 @@ crash-safe at-least-once delivery with exactly-once deduplication.
 - `MessageCodec` — TLV-based codec for message serialization/deserialization.
 - `DeliveryStore` / `TxAwareDeliveryStore` — Interfaces for durable mailbox persistence (enqueue, claim, ack, dead-letter).
 - `DurableActor` — Actor variant with crash-safe mailbox backed by SQL persistence. Provides `Wait(ctx)` to block until the actor stops and `StopAndWait(ctx)` to request a graceful shutdown and then wait.
-- `DurableActorConfig[M, R]` — Configuration struct for `DurableActor`: behavior, store, codec, clock, DLO, WaitGroup, `TellRetryPolicy`, lease/heartbeat/poll durations, max attempts, cleanup timeout, and deduplication TTL.
+- `DurableActorConfig[M, R]` — Configuration struct for `DurableActor`: behavior, store, codec, clock, DLO, WaitGroup, `TellRetryPolicy`, lease/heartbeat/poll durations, max attempts, cleanup timeout, deduplication TTL, and `NumWorkers`.
+- `DurableActorConfig.NumWorkers` — How many concurrent worker loops drain the actor's single mailbox. Default and any value `<= 1` is one worker (strictly-sequential processing). A value `> 1` turns the actor into a competing-consumer pool: that many goroutines each lease distinct messages via `LeaseNextMailboxMessage`, so independent messages run in parallel while per-correlation-key FIFO still keeps same-key messages ordered. Only for behaviors whose handlers are concurrency-safe and hold no writer across their side effects (e.g. the serverconn egress sender on the Read/Commit path). `NewDurableActor` **fails closed** with `ErrConcurrentClassicBehavior` when `NumWorkers > 1` is paired with a classic (`Left`) `ActorBehavior`, since the classic path wraps the whole `Receive` in one write transaction and assumes sequential delivery; pools are only valid on the Read/Commit (`TxBehavior`) path. The test-only `DurableActorConfig.AllowConcurrentClassicBehavior()` escape hatch bypasses the guard for the egress benchmark that measures the forbidden config; production code must never call it.
 - `DefaultDurableActorConfig[M, R]()` — Constructor returning a `DurableActorConfig` with safe defaults (30s lease, 10 max attempts, 1s poll fallback, DefaultTellRetryPolicy).
 - `TellRetryPolicy` — Function type `func(attempts int, lastErr error) (bool, time.Duration)` determining retry behavior for failed Tell messages. Return `(false, _)` to dead-letter immediately.
 - `DefaultTellRetryPolicy` — Exponential backoff policy: up to 5 attempts, starting at 1s, capped at 60s.
@@ -60,7 +61,7 @@ crash-safe at-least-once delivery with exactly-once deduplication.
 
 ## Invariants
 
-- Messages are processed sequentially per actor — no concurrent `Receive` calls.
+- Messages are processed sequentially per actor by default (one worker, no concurrent `Receive` calls). Opting into `DurableActorConfig.NumWorkers > 1` relaxes this: that many worker loops drain the one mailbox concurrently, so `Receive` may run in parallel across distinct messages. The competing-consumer lease guarantees each message is still processed by exactly one worker, and per-correlation-key FIFO holds across workers; only behaviors with concurrency-safe handlers should set it. The combination is structurally restricted to the Read/Commit path: `NewDurableActor` rejects `NumWorkers > 1` on a classic `ActorBehavior` with `ErrConcurrentClassicBehavior` so a stateful, sequentially-assumed actor can never be silently fanned out.
 - `Tell` with a `DurableActor` persists the message before returning (crash-safe enqueue).
 - Outbox messages are dispatched only after state is persisted (outbox pattern).
 - `ServiceKey` lookup via `Receptionist` is type-safe: mismatched types return `ErrServiceKeyTypeMismatch`.

db/AGENTS.md

@@ -48,8 +48,12 @@ For field-level detail, use `go doc github.com/lightninglabs/darepo-client/db.<S
   `Status`, `Trigger`, `LastError`, `SweepTxid`, `Created/UpdatedAt`.
 - `UnilateralExitJobStatus` — `Pending(0)`, `Materializing(1)`,
   `CSVPending(2)`, `Sweeping(3)`, `Completed(4)`, `Failed(5)`,
-  `SweepBroadcasting(6)`. **Append-only**: `SweepBroadcasting` is
-  last so existing rows at 3 keep decoding correctly.
+  `SweepBroadcasting(6)`, `FailedRecoverable(7)`. **Append-only**: new
+  values are added at the end so a row's numeric meaning never shifts.
+  `FailedRecoverable` is a terminal failure that left no on-chain
+  footprint, so boot-time reconciliation may roll the VTXO back to live;
+  it is excluded from `ListNonTerminalUnilateralExitJobs` alongside `4`
+  and `5` (darepo-client#602).
 - `UnilateralExitJobTrigger` — `Manual(0)`, `CriticalExpiry(1)`,
   `Restart(2)`, `FraudSpend(3)`.
 - `VHTLCRecoveryStoreDB` — durable vHTLC recovery store. Persists
@@ -67,7 +71,17 @@ For field-level detail, use `go doc github.com/lightninglabs/darepo-client/db.<S
   safety bounds enforced during `DeserializeTree`.
 - `resolveInputPackage` / `loadPackageBundleBySessionID` — two-stage
   OOR ancestry resolver (`oor_unroll_resolver.go`).
-- `LatestMigrationVersion = 16` — current schema version.
+- `LatestMigrationVersion = 17` — current schema version.
+- `SpendingReservationPersistenceStore` — Persists the durable index of VTXO
+  outpoints reserved by an active spend owner (e.g. an outgoing OOR session).
+  A row exists IFF the owning session was durably checkpointed, so a startup
+  sweep can deterministically release orphaned Spending VTXOs with no row.
+  Methods: `UpsertReservation(ctx, outpoint, ownerKind, ownerID)` (upserts a
+  row), `ListReservedOutpoints(ctx)` (returns all reserved outpoints for the
+  startup sweep). Implements both `oor.ReservationStore` and
+  `vtxo.SpendingReservationStore`.
+- `SpendingReservationStore` / `BatchedSpendingReservationStore` — Internal
+  sqlc-backed query interfaces for the reservation table.
 
 ## Relationships
 
@@ -111,6 +125,11 @@ For field-level detail, use `go doc github.com/lightninglabs/darepo-client/db.<S
 
 ### Migration notes
 
+- `000017_spending_reservations` — adds `spending_reservations` table with
+  `(outpoint_hash, outpoint_index)` PK, `owner_kind`, `owner_id`, and
+  `created_at`. A row exists IFF the owning spend session was durably
+  checkpointed. The table supports the startup orphan sweep in the VTXO
+  manager (`vtxo.Manager.sweepOrphanedReservations`).
 - `000016_unilateral_exit_policy` — adds `exit_policy_kind`
   (NOT NULL, default `'standard_vtxo_timeout'`) and nullable
   `exit_policy_ref` to `unilateral_exit_jobs` via ALTER TABLE so

db/CLAUDE.md

@@ -71,7 +71,17 @@ For field-level detail, use `go doc github.com/lightninglabs/darepo-client/db.<S
   safety bounds enforced during `DeserializeTree`.
 - `resolveInputPackage` / `loadPackageBundleBySessionID` — two-stage
   OOR ancestry resolver (`oor_unroll_resolver.go`).
-- `LatestMigrationVersion = 16` — current schema version.
+- `LatestMigrationVersion = 17` — current schema version.
+- `SpendingReservationPersistenceStore` — Persists the durable index of VTXO
+  outpoints reserved by an active spend owner (e.g. an outgoing OOR session).
+  A row exists IFF the owning session was durably checkpointed, so a startup
+  sweep can deterministically release orphaned Spending VTXOs with no row.
+  Methods: `UpsertReservation(ctx, outpoint, ownerKind, ownerID)` (upserts a
+  row), `ListReservedOutpoints(ctx)` (returns all reserved outpoints for the
+  startup sweep). Implements both `oor.ReservationStore` and
+  `vtxo.SpendingReservationStore`.
+- `SpendingReservationStore` / `BatchedSpendingReservationStore` — Internal
+  sqlc-backed query interfaces for the reservation table.
 
 ## Relationships
 
@@ -115,6 +125,11 @@ For field-level detail, use `go doc github.com/lightninglabs/darepo-client/db.<S
 
 ### Migration notes
 
+- `000017_spending_reservations` — adds `spending_reservations` table with
+  `(outpoint_hash, outpoint_index)` PK, `owner_kind`, `owner_id`, and
+  `created_at`. A row exists IFF the owning spend session was durably
+  checkpointed. The table supports the startup orphan sweep in the VTXO
+  manager (`vtxo.Manager.sweepOrphanedReservations`).
 - `000016_unilateral_exit_policy` — adds `exit_policy_kind`
   (NOT NULL, default `'standard_vtxo_timeout'`) and nullable
   `exit_policy_ref` to `unilateral_exit_jobs` via ALTER TABLE so

ledger/AGENTS.md

@@ -35,9 +35,13 @@ For per-flow walkthroughs see
 
 For field-level detail, use `go doc github.com/lightninglabs/darepo-client/ledger.<Symbol>`.
 
-- `LedgerActor` — durable actor processing accounting messages. Caches
-  the resolved `clock.Clock` at construction so handlers stamp
-  `CreatedAt` without re-optioning the field.
+- `LedgerActor` — durable actor processing accounting messages. Runs on the
+  durable Read/Commit (`TxBehavior`) path: each handler books its ledger legs
+  inside one short, lease-fenced Commit transaction rather than holding a
+  writer tx across the whole `Receive`. The `bindStores` factory injects a
+  `ledgerTx` (typed store pair) bound to each Commit transaction. Caches the
+  resolved `clock.Clock` at construction so handlers stamp `CreatedAt` without
+  re-optioning the field.
 - `ActorConfig` — logger, delivery store, ledger store, UTXO audit
   store, actor ID, optional `Clock` (`fn.Option[clock.Clock]`); None
   falls back to `clock.NewDefaultClock()`.

ledger/CLAUDE.md

@@ -35,9 +35,13 @@ For per-flow walkthroughs see
 
 For field-level detail, use `go doc github.com/lightninglabs/darepo-client/ledger.<Symbol>`.
 
-- `LedgerActor` — durable actor processing accounting messages. Caches
-  the resolved `clock.Clock` at construction so handlers stamp
-  `CreatedAt` without re-optioning the field.
+- `LedgerActor` — durable actor processing accounting messages. Runs on the
+  durable Read/Commit (`TxBehavior`) path: each handler books its ledger legs
+  inside one short, lease-fenced Commit transaction rather than holding a
+  writer tx across the whole `Receive`. The `bindStores` factory injects a
+  `ledgerTx` (typed store pair) bound to each Commit transaction. Caches the
+  resolved `clock.Clock` at construction so handlers stamp `CreatedAt` without
+  re-optioning the field.
 - `ActorConfig` — logger, delivery store, ledger store, UTXO audit
   store, actor ID, optional `Clock` (`fn.Option[clock.Clock]`); None
   falls back to `clock.NewDefaultClock()`.

oor/AGENTS.md

@@ -29,6 +29,12 @@ State transitions and validation rules live under [Invariants](#invariants).
   materialized — lets daemon subsystems arm work without depending on
   `oor`), `SigningEffect` (route signing through a separate actor),
   `Limits *ReceiveLimits` (defaults via `DefaultReceiveLimits`).
+- `ReservationStore` — Minimal persistence contract for durable spending
+  reservations. `UpsertReservation(ctx, outpoint, ownerKind, ownerID)` is
+  called once a new outgoing OOR session is checkpointed, so the startup VTXO
+  sweep can tell in-flight spends from orphaned ones.
+  `ReservationOwnerKindOOROutgoing = 0` is the owner-kind value recorded for
+  outgoing OOR sessions.
 - `OORClientActor` — durable actor wrapping per-session state machines.
   Handles outgoing and incoming flows via three-phase async resolution;
   emits `VTXOSentMsg` / `VTXOReceivedMsg` to ledger at the two state
@@ -130,6 +136,11 @@ State transitions and validation rules live under [Invariants](#invariants).
   `completed`, `failed`).
 - `IncomingSnapshot`, `IncomingPhase` (`resolve_pending`,
   `materialize_pending`, `ack_pending`, `completed`, `failed`).
+  `IncomingSnapshot.MetadataAttempts uint32` — persisted retry count for
+  authoritative metadata resolution (phase-2 indexer query). Drives bounded
+  exponential backoff and terminal give-up in `handleReceiveOutboxError`
+  across restarts so a session whose VTXO never lands in the indexer stops
+  re-querying forever. Serialized as TLV record 19.
 - `TransferInputSnapshot` — portable encoding of client-side signing
   context required to finalize checkpoint PSBTs after restart.
 - `IncomingVTXOMetadata` — lineage metadata for incoming OOR VTXOs

oor/CLAUDE.md

@@ -29,6 +29,12 @@ State transitions and validation rules live under [Invariants](#invariants).
   materialized — lets daemon subsystems arm work without depending on
   `oor`), `SigningEffect` (route signing through a separate actor),
   `Limits *ReceiveLimits` (defaults via `DefaultReceiveLimits`).
+- `ReservationStore` — Minimal persistence contract for durable spending
+  reservations. `UpsertReservation(ctx, outpoint, ownerKind, ownerID)` is
+  called once a new outgoing OOR session is checkpointed, so the startup VTXO
+  sweep can tell in-flight spends from orphaned ones.
+  `ReservationOwnerKindOOROutgoing = 0` is the owner-kind value recorded for
+  outgoing OOR sessions.
 - `OORClientActor` — durable actor wrapping per-session state machines.
   Handles outgoing and incoming flows via three-phase async resolution;
   emits `VTXOSentMsg` / `VTXOReceivedMsg` to ledger at the two state
@@ -130,6 +136,11 @@ State transitions and validation rules live under [Invariants](#invariants).
   `completed`, `failed`).
 - `IncomingSnapshot`, `IncomingPhase` (`resolve_pending`,
   `materialize_pending`, `ack_pending`, `completed`, `failed`).
+  `IncomingSnapshot.MetadataAttempts uint32` — persisted retry count for
+  authoritative metadata resolution (phase-2 indexer query). Drives bounded
+  exponential backoff and terminal give-up in `handleReceiveOutboxError`
+  across restarts so a session whose VTXO never lands in the indexer stops
+  re-querying forever. Serialized as TLV record 19.
 - `TransferInputSnapshot` — portable encoding of client-side signing
   context required to finalize checkpoint PSBTs after restart.
 - `IncomingVTXOMetadata` — lineage metadata for incoming OOR VTXOs

p-models/durableactor/AGENTS.md

@@ -2,8 +2,9 @@
 
 This package models the durable actor mailbox from distributed-systems first
 principles: durable enqueue, lease ownership, retry scheduling, ack/nack token
-validation, dead-letter/removal, idempotent delivery identity, and
-per-correlation-key FIFO.
+validation, dead-letter/removal, idempotent delivery identity,
+per-correlation-key FIFO, and the Read/Commit consume step (lease-fenced
+exactly-once effect application under lease-expiry-during-IO).
 
 ## Files
 
@@ -22,7 +23,12 @@ per-correlation-key FIFO.
 | `./p-models/scripts/check.sh` | Full default check: P model plus Go bridge |
 | `p compile -pp p-models/durableactor/infra.pproj` | Compile this model |
 | `p check PGenerated/PChecker/net8.0/MailboxInfraModels.dll --testcase tcMailboxCorrelationKeyFIFO` | Run green durable mailbox tests |
+| `p check PGenerated/PChecker/net8.0/MailboxInfraModels.dll --testcase tcMailboxReadCommitFence` | Run the green Read/Commit exactly-once-effect test |
 | `p check PGenerated/PChecker/net8.0/MailboxInfraModels.dll --testcase tcMailboxLegacyReorderCounterexample` | Demonstrate the old ordering bug |
+| `p check PGenerated/PChecker/net8.0/MailboxInfraModels.dll --testcase tcMailboxUnfencedCommitCounterexample` | Demonstrate the unfenced-commit double-apply bug |
+| `p check PGenerated/PChecker/net8.0/MailboxInfraModels.dll --testcase tcMailboxStageCommitExactlyOnce` | Run the green Stage-then-Commit replay-safety test |
+| `p check PGenerated/PChecker/net8.0/MailboxInfraModels.dll --testcase tcMailboxStagedDoubleBroadcastCounterexample` | Demonstrate the unstable-broadcast double-broadcast bug |
+| `p check PGenerated/PChecker/net8.0/MailboxInfraModels.dll --testcase tcMailboxStaleStageRegressesCounterexample` | Demonstrate the unfenced-stage checkpoint regression bug |
 | `go test ./p-models/durableactor/bridge` | Replay traces against Go |
 
 ## Modeling Guidance

p-models/durableactor/bridge/AGENTS.md

@@ -0,0 +1,55 @@
+# p-models/durableactor/bridge
+
+## Purpose
+
+Go conformance harness that replays P model mailbox traces against the real
+`db/actordelivery` SQLite store. Keeps the formal P model abstraction tied to
+the SQL claim implementation: every P scenario in `mailbox_fifo_test.p`
+that produces a trace is replayed here using production store code, so a
+divergence between the model and the implementation fails the Go test rather
+than the P checker.
+
+## Key Types
+
+- `MailboxTrace` — A named sequence of mailbox operations loaded from a JSON
+  trace file (`trace_id`, `description`, `events`).
+- `MailboxTraceEvent` — One store operation in a trace: `op` (enqueue/lease/
+  nack/ack/commit/dead_letter/expire_leases), plus op-specific fields for id,
+  mailbox_id, lease_token, expected outcome, etc. `ExpectDuplicate` asserts
+  idempotent no-op enqueue semantics. `ExpectProcessed` verifies the dedup
+  mark after a fenced commit.
+- `ParseMailboxTrace(path)` — Parses one trace file from disk.
+- `ParseMailboxTraceDir(dir)` — Parses all `*.json` trace files in a directory,
+  sorted by `TraceID`.
+- `ReplayMailboxTrace(t, trace)` — Replays a trace against a fresh SQLite
+  `actordelivery` store in a temp dir. The `commit` op models the Read/Commit
+  fenced-ack pattern exactly: it runs `AckMessage` + `MarkProcessed` inside one
+  writer transaction, rolling back with `actor.ErrLeaseLost` when the ack row
+  count is zero.
+
+## Relationships
+
+- **Depends on**: `db/actordelivery` (real store under test), `baselib/actor`
+  (store interfaces, `ErrLeaseLost`), `db/sqlc` (backend type constants).
+- **Depended on by**: nothing (test-only package, invoked via
+  `go test ./p-models/durableactor/bridge`).
+
+## Invariants
+
+- Every trace op that can fail hard uses `t.Fatal`; partial replays are not
+  allowed to proceed silently.
+- The `commit` op is the sole site where `ExecTx`/`AckMessage`/`MarkProcessed`
+  are combined — it deliberately mirrors `execCore.commit` in `baselib/actor`
+  so the P model's commit-fence scenario stays tied to the real SQL path.
+- Duplicate enqueue ops (`ExpectDuplicate: true`) must complete without error;
+  a future rejection would fail here explicitly rather than at a later lease
+  step.
+
+## Deep Docs
+
+- [p-models/durableactor/CLAUDE.md](../CLAUDE.md) — P model structure, trace
+  layout, and check commands.
+- [p-models/CLAUDE.md](../../CLAUDE.md) — Top-level p-models layout and
+  orchestration.
+- [docs/durable_actor_architecture.md](../../../docs/durable_actor_architecture.md)
+  — Durable actor internals.