Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 9 additions & 6 deletions darepod/server.go
Original file line number Diff line number Diff line change
Expand Up @@ -5014,12 +5014,15 @@ func (s *Server) initUnrollSubsystem(ctx context.Context,
Store: &unroll.DBRegistryStore{
UEStore: ueStore,
},
DeliveryStore: s.deliveryStore,
ProofAssembler: proofAssembler,
VTXOStore: vtxoStore,
TxConfirmRef: unrollTxConfirmRef,
ChainSource: chainSourceRef,
Wallet: unrollWallet,
DeliveryStore: s.deliveryStore,
ProofAssembler: proofAssembler,
VTXOStore: vtxoStore,
TxConfirmRef: unrollTxConfirmRef,
ChainSource: chainSourceRef,
Wallet: unrollWallet,
LedgerSink: fn.Some(
ledger.NewSink(s.actorSystem),
),
Log: fn.Some(s.subLogger("UNRL")),
MaxSweepFeeRateSatPerVByte: s.unrollMaxFeeRate(),
ExitSpendPolicyResolver: unrollpolicy.ExitSpendPolicyResolver{
Expand Down
35 changes: 33 additions & 2 deletions db/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,22 @@ For field-level detail, use `go doc github.com/lightninglabs/darepo-client/db.<S
safety bounds enforced during `DeserializeTree`.
- `resolveInputPackage` / `loadPackageBundleBySessionID` — two-stage
OOR ancestry resolver (`oor_unroll_resolver.go`).
- `LatestMigrationVersion = 17` — current schema version.
- `LatestMigrationVersion = 20` — current schema version.
- `PendingIntentPersistenceStore` — implements `wallet.PendingIntentStore`,
the persistence half of the generic restart-safe intent outbox (header
`pending_intents` + per-kind detail tables + `pending_intent_anchors`).
Maps the sealed `wallet.PendingIntentPayload` concrete types to/from typed
detail columns (no blob). Intents are written before the wallet publishes
them to the round actor; `CommitState` clears anchors by outpoint
(boarding outpoints AND forfeited VTXO outpoints) inside the
point-of-no-return round checkpoint transaction, then sweeps orphaned
detail rows and headers, so replay-after-adoption is structurally
impossible. Methods: `UpsertPendingIntent` (header + detail + anchors
atomically; anchor rebind sweeps anchor-less older intents),
`ListPendingIntents` (per kind, with anchors), `DeletePendingIntent`,
`ClearPendingIntentsByKind`.
- `PendingIntentStore` / `BatchedPendingIntentStore` — internal sqlc-backed
query interfaces for the pending-intent tables.
- `SpendingReservationPersistenceStore` — Persists the durable index of VTXO
outpoints reserved by an active spend owner (e.g. an outgoing OOR session).
A row exists IFF the owning session was durably checkpointed, so a startup
Expand Down Expand Up @@ -125,6 +140,22 @@ For field-level detail, use `go doc github.com/lightninglabs/darepo-client/db.<S

### Migration notes

- `000020_accounting_wallet_sweeps` — adds `wallet_clearing`,
`wallet_utxo_spent`, and `wallet_sweep_transfer` for sweep
accounting. Rebuilds the round idempotency index so keyed
round events use `idx_client_ledger_idempotent_key` instead of
collapsing on `(round_id, event_type, debit_account, credit_account)`.
Renumbered from 000019 to land after `000019_oor_session_registry`,
which merged to main while this work was in review.
- `000018_pending_intents` — generalizes the Board-only
`pending_board_requests` outbox into a supertype/subtype set:
`pending_intent_kinds` (enum table), `pending_intents` (header: 32-byte
hash-derived intent id + kind FK + requested_at, no payload blob),
per-kind detail tables `pending_board_intents` / `pending_send_intents`
with first-class typed columns, and `pending_intent_anchors` (one row per
anchored outpoint, PK on the outpoint so a newer intent rebinds, FK to the
header). Drops `pending_board_requests` outright (alpha; rows only exist
in the narrow crash window between admission and round seal).
- `000017_spending_reservations` — adds `spending_reservations` table with
`(outpoint_hash, outpoint_index)` PK, `owner_kind`, `owner_id`, and
`created_at`. A row exists IFF the owning spend session was durably
Expand All @@ -147,7 +178,7 @@ For field-level detail, use `go doc github.com/lightninglabs/darepo-client/db.<S
- `000013_pending_board_request` — records the user's explicit `Board`
RPC intent so a daemon restart between Board admission and round
seal does not silently drop the request. Keyed by the confirmed
boarding outpoint.
boarding outpoint. Superseded by `000018_pending_intents`.
- `000012_boarding_sweep_ledger_events` — registers the
`boarding_sweep_fee_paid` ledger event type so `FeePaidMsg` with
`FeeType=FeeTypeOnchainSweep` satisfies the `ledger_entries.event_type`
Expand Down
9 changes: 8 additions & 1 deletion db/CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ For field-level detail, use `go doc github.com/lightninglabs/darepo-client/db.<S
safety bounds enforced during `DeserializeTree`.
- `resolveInputPackage` / `loadPackageBundleBySessionID` — two-stage
OOR ancestry resolver (`oor_unroll_resolver.go`).
- `LatestMigrationVersion = 19` — current schema version.
- `LatestMigrationVersion = 20` — current schema version.
- `PendingIntentPersistenceStore` — implements `wallet.PendingIntentStore`,
the persistence half of the generic restart-safe intent outbox (header
`pending_intents` + per-kind detail tables + `pending_intent_anchors`).
Expand Down Expand Up @@ -140,6 +140,13 @@ For field-level detail, use `go doc github.com/lightninglabs/darepo-client/db.<S

### Migration notes

- `000020_accounting_wallet_sweeps` — adds `wallet_clearing`,
`wallet_utxo_spent`, and `wallet_sweep_transfer` for sweep
accounting. Rebuilds the round idempotency index so keyed
round events use `idx_client_ledger_idempotent_key` instead of
collapsing on `(round_id, event_type, debit_account, credit_account)`.
Renumbered from 000019 to land after `000019_oor_session_registry`,
which merged to main while this work was in review.
- `000018_pending_intents` — generalizes the Board-only
`pending_board_requests` outbox into a supertype/subtype set:
`pending_intent_kinds` (enum table), `pending_intents` (header: 32-byte
Expand Down
11 changes: 8 additions & 3 deletions db/ledger_store_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -1327,10 +1327,10 @@ func TestLedgerStoreListAccounts(t *testing.T) {
accounts, err := store.ListAccounts(ctx)
require.NoError(t, err)

// The migration seeds 7 accounts: wallet_balance, vtxo_balance,
// The migrations seed 8 accounts: wallet_balance, vtxo_balance,
// fees_paid, onchain_fees, transfers_in, transfers_out,
// opening_balance.
require.Len(t, accounts, 7)
// opening_balance, wallet_clearing.
require.Len(t, accounts, 8)

// Build a map for easier assertions.
byID := make(map[string]sqlc.Account, len(accounts))
Expand Down Expand Up @@ -1359,6 +1359,11 @@ func TestLedgerStoreListAccounts(t *testing.T) {
require.Equal(
t, "Opening Balance", byID["opening_balance"].AccountName,
)

require.Equal(t, "asset", byID["wallet_clearing"].AccountType)
require.Equal(
t, "Wallet Sweep Clearing", byID["wallet_clearing"].AccountName,
)
}

// TestLedgerStoreIdempotentInsert verifies that a redelivered
Expand Down
2 changes: 1 addition & 1 deletion db/migrations.go
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ const (
// daemon.
//
// NOTE: This MUST be updated when a new migration is added.
LatestMigrationVersion uint = 19
LatestMigrationVersion uint = 20
)

// MigrationTarget is a functional option that can be passed to applyMigrations
Expand Down
112 changes: 112 additions & 0 deletions db/sqlc/fee_accounting.sql.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

10 changes: 10 additions & 0 deletions db/sqlc/migrations/000020_accounting_wallet_sweeps.down.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
DROP INDEX IF EXISTS idx_client_ledger_idempotent_round;
CREATE UNIQUE INDEX IF NOT EXISTS idx_client_ledger_idempotent_round
ON ledger_entries(round_id, event_type, debit_account, credit_account)
WHERE round_id IS NOT NULL;

DELETE FROM ledger_event_types
WHERE event_type IN ('wallet_utxo_spent', 'wallet_sweep_transfer');

DELETE FROM accounts
WHERE account_id = 'wallet_clearing';
21 changes: 21 additions & 0 deletions db/sqlc/migrations/000020_accounting_wallet_sweeps.up.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
-- Add the clearing account and event types needed to account for
-- wallet-level sweeps without double-counting internal return outputs.
INSERT INTO accounts (account_id, account_name, account_type) VALUES
('wallet_clearing', 'Wallet Sweep Clearing', 'asset')
ON CONFLICT DO NOTHING;

INSERT INTO ledger_event_types (event_type) VALUES
('wallet_utxo_spent'),
('wallet_sweep_transfer')
ON CONFLICT DO NOTHING;

-- Round-scoped events without a specific key remain deduped by
-- (round_id, event_type, accounts). Events that carry an explicit
-- idempotency_key, such as per-recipient round sends, are instead
-- deduped by idx_client_ledger_idempotent_key so multiple sends in
-- the same round can coexist.
DROP INDEX IF EXISTS idx_client_ledger_idempotent_round;
CREATE UNIQUE INDEX IF NOT EXISTS idx_client_ledger_idempotent_round
ON ledger_entries(round_id, event_type, debit_account, credit_account)
WHERE round_id IS NOT NULL
AND idempotency_key IS NULL;
3 changes: 3 additions & 0 deletions db/sqlc/querier.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

32 changes: 32 additions & 0 deletions db/sqlc/queries/fee_accounting.sql
Original file line number Diff line number Diff line change
Expand Up @@ -290,3 +290,35 @@ SELECT COUNT(*) FROM ledger_entries;
SELECT account_id, account_name, account_type
FROM accounts
ORDER BY account_id;

-- name: ListClientAccountBalances :many
SELECT a.account_id,
a.account_name,
a.account_type,
CAST(COALESCE(SUM(
CASE
WHEN le.debit_account = a.account_id THEN le.amount_sat
WHEN le.credit_account = a.account_id THEN -le.amount_sat
ELSE CAST(0 AS BIGINT)
END
), 0) AS BIGINT) AS balance_sat
FROM accounts AS a
LEFT JOIN ledger_entries AS le
ON le.debit_account = a.account_id
OR le.credit_account = a.account_id
GROUP BY a.account_id, a.account_name, a.account_type
ORDER BY a.account_id;
Comment on lines +294 to +310

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Using an OR condition in a LEFT JOIN (e.g., le.debit_account = a.account_id OR le.credit_account = a.account_id) prevents SQLite from efficiently utilizing indexes on debit_account and credit_account, resulting in full table scans. Since ledger_entries can grow very large in production, this query will become a performance bottleneck. Rewriting the query to use two correlated subqueries allows SQLite to perform highly efficient index lookups.

-- name: ListClientAccountBalances :many
SELECT a.account_id,
       a.account_name,
       a.account_type,
       CAST(
           COALESCE((SELECT SUM(amount_sat) FROM ledger_entries WHERE debit_account = a.account_id), 0) -
           COALESCE((SELECT SUM(amount_sat) FROM ledger_entries WHERE credit_account = a.account_id), 0)
       AS BIGINT) AS balance_sat
FROM accounts AS a
ORDER BY a.account_id;


-- name: ListClientLedgerEventTotals :many
SELECT event_type,
CAST(COUNT(*) AS BIGINT) AS entry_count,
CAST(COALESCE(SUM(amount_sat), 0) AS BIGINT) AS total_sat
FROM ledger_entries
GROUP BY event_type
ORDER BY event_type;

-- name: GetClientLedgerStats :one
SELECT CAST(COUNT(*) AS BIGINT) AS entry_count,
CAST(COALESCE(MIN(created_at), 0) AS BIGINT) AS first_created_at,
CAST(COALESCE(MAX(created_at), 0) AS BIGINT) AS last_created_at
FROM ledger_entries;
3 changes: 2 additions & 1 deletion db/sqlc/schemas/generated_schema.sql
Original file line number Diff line number Diff line change
Expand Up @@ -286,7 +286,8 @@ CREATE UNIQUE INDEX idx_client_ledger_idempotent_key

CREATE UNIQUE INDEX idx_client_ledger_idempotent_round
ON ledger_entries(round_id, event_type, debit_account, credit_account)
WHERE round_id IS NOT NULL;
WHERE round_id IS NOT NULL
AND idempotency_key IS NULL;

CREATE UNIQUE INDEX idx_client_ledger_idempotent_session
ON ledger_entries(session_id, event_type, debit_account, credit_account)
Expand Down
Loading
Loading