Incremental commits off-chain (#1541)

cardano-scaling · Oct 4, 2024 · 4d48208 · 4d48208
2 parents ff30a60 + efe7083
commit 4d48208
Show file tree

Hide file tree

Showing 82 changed files with 62,104 additions and 22,056 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -10,6 +10,17 @@ changes.
 
 ## Unreleased
 
+- **IMPORTANT - Do not release this version**
+   - Incremental commits - off-chain changes to make the incremental commits possible.
+     Important to note is that on-chain security is not implemented and hydra-node in this
+     state is not releasable!
+     Missing off-chain items to implement as a series of next PR's:
+      - Use the observed UTxO to construct increment/recover
+      - Add documentation to explain the feature
+      - Implement tests scenarios outlined in the [incremental-commit] (https://github.com/cardano-scaling/hydra/issues/199) issue
+      - Remove Commit client input since it is unused
+      - Revisit types related to observations/posting transactions and make sure the fields are named appropriatelly
+
 - Tested with `cardano-node 9.2.0` and `cardano-cli 9.4.1.0`.
 
 ## [0.19.0] - 2024-09-13

diff --git a/docs/docs/dev/protocol.md b/docs/docs/dev/protocol.md
@@ -2,42 +2,98 @@
 
 Additional implementation-specific documentation for the Hydra Head protocol and extensions like incremental decommits.
 
+### Incremental Commits
+
+#### Deposit flow
+
+```mermaid
+sequenceDiagram
+    Alice->>+Node A: POST /commit UTxO
+    Node A-->>-Alice: depositTx
+
+    Alice ->> Alice: sign depositTx
+    Alice ->> Chain: submit depositTx
+
+    Chain ->>+ Node A: OnDepositTx utxo
+    Chain ->>+ Node B: OnDepositTx utxo
+
+    Node A -->> Alice: CommitRecorded
+
+    par Alice isLeader
+        Node A->>Node A: ReqSn utxoToCommit
+    and
+        Node A->>Node B: ReqSn utxoToCommit
+    end
+
+    Node A->>Node A: sig = sign snapshot incl. utxoToCommit
+
+    par broadcast
+        Node A->>Node A: AckSn sig
+    and
+        Node A->>Node B: AckSn sig
+    end
+    Node B->>Node A: AckSn sig
+    Node A -->> Alice: SnapshotConfirmed
+
+    Node A -->> Alice: CommitApproved
+
+    Node A ->> Chain: IncrementTx snapshot sig
+    Chain ->> Node A: OnIncrementTx
+    Node A -->> Alice: CommitFinalized
+```
+
+#### Recover flow
+
+```mermaid
+sequenceDiagram
+    Alice->>+Node A: DELETE /commits/<tx-id>
+    Node A->>Chain: recoverTx
+    Chain ->>+ Node A: OnRecoverTx utxo
+    Chain ->>+ Node B: OnRecoverTx utxo
+    Node A -->>- Alice: CommitRecovered
+    Node B -->>- Bob: CommitRecovered
+    Node A-->>-Alice: OK
+
+```
+
 ### Incremental Decommits
 
 ```mermaid
 sequenceDiagram
-    Alice->>HeadLogic: Decommit (decTx)
-    HeadLogic->>HeadLogic: canApply decTx
+    Alice->>+Node A: POST /decommit (decTx)
+    Node A-->>-Alice: OK
+
+    Node A->>Node A: canApply decTx
 
     par broadcast
-        HeadLogic->>HeadLogic: ReqDec decTx
+        Node A->>Node A: ReqDec decTx
     and
-        HeadLogic->>Node B: ReqDec decTx
+        Node A->>Node B: ReqDec decTx
     end
 
-    HeadLogic -->> Alice: DecommitRequested
+    Node A -->> Alice: DecommitRequested
 
     par Alice isLeader
-        HeadLogic->>HeadLogic: ReqSn decTx
+        Node A->>Node A: ReqSn decTx
     and
-        HeadLogic->>Node B: ReqSn decTx
+        Node A->>Node B: ReqSn decTx
     end
 
-    HeadLogic->>HeadLogic: canApply decTx, decUTxO =  outputs(decTx)
-    HeadLogic->>HeadLogic: sig = sign snapshot incl. decUTxO
+    Node A->>Node A: canApply decTx, decUTxO =  outputs(decTx)
+    Node A->>Node A: sig = sign snapshot incl. decUTxO
 
     par broadcast
-        HeadLogic->>HeadLogic: AckSn sig
+        Node A->>Node A: AckSn sig
     and
-        HeadLogic->>Node B: AckSn sig
+        Node A->>Node B: AckSn sig
     end
 
-    Node B->>HeadLogic: AckSn sig
+    Node B->>Node A: AckSn sig
 
-    HeadLogic -->> Alice: SnapshotConfirmed
-    HeadLogic -->> Alice: DecommitApproved
+    Node A -->> Alice: SnapshotConfirmed
+    Node A -->> Alice: DecommitApproved
 
-    HeadLogic ->> Chain: DecrementTx snapshot sig
-    Chain ->> HeadLogic: OnDecrementTx
-    HeadLogic -->> Alice: DecommitFinalized
+    Node A ->> Chain: DecrementTx snapshot sig
+    Chain ->> Node A: OnDecrementTx
+    Node A -->> Alice: DecommitFinalized
 ```
diff --git a/docs/docs/how-to/incremental-commit.md b/docs/docs/how-to/incremental-commit.md
@@ -0,0 +1,102 @@
+# Commit funds to an open Head
+
+Assuming we already have an open Head and some funds on the L1 we would like to commit.
+
+
+:::info
+You could run a local [demo](./../getting-started)
+:::
+
+The following commands are expected to be run from the `demo` folder and using these environment variables:
+
+```shell
+export CARDANO_NODE_SOCKET_PATH=${PWD}/devnet/node.socket
+export CARDANO_NODE_NETWORK_ID=42
+```
+
+We can inspect the L1 utxo with:
+
+```shell
+cardano-cli query utxo --whole-utxo
+```
+and the state of the faucet public key
+
+```shell
+cardano-cli query utxo \
+  --address $(cardano-cli address build --payment-verification-key-file ${PWD}/../hydra-cluster/config/credentials/faucet.vk)
+```
+
+
+In this setup, we would be using the `faucet` keys to commit everything into the head.
+
+```shell
+export WALLET_SK=${PWD}/../hydra-cluster/config/credentials/faucet.sk
+export WALLET_VK=${PWD}/../hydra-cluster/config/credentials/faucet.vk
+```
+
+### Deposit UTxO to commit
+
+The `/commit` endpoint supports two ways of specifying what to commit, one is just by showing the UTxO (which is assumed to be owned by public keys), while the more advanced way would be using [blueprint transactions](./commit-blueprint).
+
+We are using the simple request here and want to commit everything owned by `${WALLET_SK}`. So we can just query the L1 for all UTxO owned by it:
+```shell
+cardano-cli query utxo \
+  --address $(cardano-cli address build --payment-verification-key-file ${WALLET_VK}) \
+  --out-file commit-utxo.json
+```
+
+Then a request to the `/commit` endpoint provides us with a transaction:
+
+```shell
+curl -X POST localhost:4001/commit \
+  --data @commit-utxo.json \
+  > deposit-tx.json
+```
+
+Which we can submit to the cardano network:
+```shell
+cardano-cli transaction sign \
+  --tx-file deposit-tx.json \
+  --signing-key-file ${WALLET_SK} \
+  --out-file deposit-tx.signed.json
+
+cardano-cli transaction submit \
+  --tx-file deposit-tx.signed.json
+```
+
+This will result in a deposit being detected by the `hydra-node` and consequently the funds to be committed to the head.
+
+### Recover a deposit
+
+Do the same thing as above, **but** with one node stopped, so the deposit is not going to be picked up.
+
+Once we deposited funds we should not see the corresponding UTxO belonging to faucet public key on L1:
+
+```shell
+
+cardano-cli query utxo \
+  --address $(cardano-cli address build --payment-verification-key-file ${PWD}/../hydra-cluster/config/credentials/faucet.vk)
+```
+
+
+Inspect the pending deposits:
+
+```
+curl -X GET localhost:4001/commits
+
+```
+and you should see the tx-id of the deposit transaction `["6b51f3787f5482004b258c60fe0c94775164f547d9284b6233bbb4f6f8b9dfa6"]`
+
+To recover, we can use the `/commits` endpoint again using the transaction id of the deposit:
+
+```shell
+curl -X DELETE localhost:4001/commits/$(printf "\"6b51f3787f5482004b258c60fe0c94775164f547d9284b6233bbb4f6f8b9dfa6"\" | jq -sRr '@uri')
+```
+
+If we inspect the faucet funds again we will see that the locked deposit is now recovered
+
+```shell
+cardano-cli query utxo \
+  --address $(cardano-cli address build --payment-verification-key-file ${PWD}/../hydra-cluster/config/credentials/faucet.vk)
+```
+
diff --git a/hydra-cardano-api/src/Hydra/Cardano/Api.hs b/hydra-cardano-api/src/Hydra/Cardano/Api.hs
@@ -114,6 +114,7 @@ import Cardano.Api.UTxO (
   UTxO' (..),
  )
 import Cardano.Ledger.Coin as X (Coin (..))
+import Hydra.Cardano.Api.Network as X (networkIdToNetwork)
 import Hydra.Cardano.Api.Prelude (
   Era,
   LedgerEra,

diff --git a/hydra-chain-observer/hydra-chain-observer.cabal b/hydra-chain-observer/hydra-chain-observer.cabal
@@ -93,6 +93,7 @@ test-suite tests
   type:               exitcode-stdio-1.0
   build-depends:
     , hspec
+    , hydra-cardano-api
     , hydra-chain-observer
     , hydra-node
     , hydra-prelude

diff --git a/hydra-chain-observer/src/Hydra/ChainObserver.hs b/hydra-chain-observer/src/Hydra/ChainObserver.hs
@@ -91,6 +91,9 @@ data ChainObserverLog
   | HeadInitTx {headId :: HeadId}
   | HeadCommitTx {headId :: HeadId}
   | HeadCollectComTx {headId :: HeadId}
+  | HeadDepositTx {headId :: HeadId}
+  | HeadRecoverTx {headId :: HeadId}
+  | HeadIncrementTx {headId :: HeadId}
   | HeadDecrementTx {headId :: HeadId}
   | HeadCloseTx {headId :: HeadId}
   | HeadFanoutTx {headId :: HeadId}
@@ -203,6 +206,9 @@ chainSyncClient tracer networkId startingPoint observerHandler =
     OnInitTx{headId} -> HeadInitTx{headId}
     OnCommitTx{headId} -> HeadCommitTx{headId}
     OnCollectComTx{headId} -> HeadCollectComTx{headId}
+    OnIncrementTx{headId} -> HeadIncrementTx{headId}
+    OnDepositTx{headId} -> HeadDepositTx{headId}
+    OnRecoverTx{headId} -> HeadRecoverTx{headId}
     OnDecrementTx{headId} -> HeadDecrementTx{headId}
     OnCloseTx{headId} -> HeadCloseTx{headId}
     OnFanoutTx{headId} -> HeadFanoutTx{headId}

diff --git a/hydra-chain-observer/test/Hydra/ChainObserverSpec.hs b/hydra-chain-observer/test/Hydra/ChainObserverSpec.hs
@@ -3,6 +3,7 @@ module Hydra.ChainObserverSpec where
 import Hydra.Prelude
 import Test.Hydra.Prelude
 
+import Hydra.Cardano.Api (utxoFromTx)
 import Hydra.Chain.Direct.State (HasKnownUTxO (getKnownUTxO), genChainStateWithTx)
 import Hydra.Chain.Direct.State qualified as Transition
 import Hydra.Chain.Direct.Tx (HeadObservation (..))
@@ -17,14 +18,15 @@ spec =
   parallel $ do
     prop "All valid transitions for all possible states can be observed." $
       checkCoverage $
-        forAllBlind genChainStateWithTx $ \(_ctx, st, tx, transition) ->
+        forAllBlind genChainStateWithTx $ \(_ctx, st, additionalUTxO, tx, transition) ->
           genericCoverTable [transition] $
             counterexample (show transition) $
-              let utxo = getKnownUTxO st
+              let utxo = getKnownUTxO st <> utxoFromTx tx <> additionalUTxO
                in case snd $ observeTx testNetworkId utxo tx of
                     Just (Init{}) -> transition === Transition.Init
                     Just (Commit{}) -> transition === Transition.Commit
                     Just (CollectCom{}) -> transition === Transition.Collect
+                    Just (Increment{}) -> transition === Transition.Increment
                     Just (Decrement{}) -> transition === Transition.Decrement
                     Just (Abort{}) -> transition === Transition.Abort
                     Just (Close{}) -> transition === Transition.Close
@@ -33,8 +35,8 @@ spec =
                     _ -> property False
 
     prop "Updates UTxO state given transaction part of Head lifecycle" $
-      forAllBlind genChainStateWithTx $ \(_ctx, st, tx, _transition) ->
-        let utxo = getKnownUTxO st
+      forAllBlind genChainStateWithTx $ \(_ctx, st, additionalUTxO, tx, _transition) ->
+        let utxo = getKnownUTxO st <> additionalUTxO
          in fst (observeTx testNetworkId utxo tx) =/= utxo
 
     prop "Does not updates UTxO state given transactions outside of Head lifecycle" $

diff --git a/hydra-cluster/hydra-cluster.cabal b/hydra-cluster/hydra-cluster.cabal
@@ -85,7 +85,7 @@ library
   build-depends:
     , aeson
     , async
-    , base                  >=4.7 && <5
+    , base                                         >=4.7 && <5
     , bytestring
     , cardano-slotting
     , containers
@@ -94,6 +94,7 @@ library
     , directory
     , filepath
     , http-conduit
+    , http-types
     , hydra-cardano-api
     , hydra-node
     , hydra-prelude
@@ -105,6 +106,7 @@ library
     , lens
     , lens-aeson
     , optparse-applicative
+    , plutus-ledger-api:plutus-ledger-api-testlib
     , process
     , QuickCheck
     , req

diff --git a/hydra-cluster/src/Hydra/Cluster/Faucet.hs b/hydra-cluster/src/Hydra/Cluster/Faucet.hs
@@ -145,18 +145,18 @@ createOutputAtAddress ::
   RunningNode ->
   AddressInEra ->
   TxOutDatum CtxTx ->
+  Value ->
   IO (TxIn, TxOut CtxUTxO)
-createOutputAtAddress node@RunningNode{networkId, nodeSocket} atAddress datum = do
+createOutputAtAddress node@RunningNode{networkId, nodeSocket} atAddress datum val = do
   (faucetVk, faucetSk) <- keysFor Faucet
-  -- we don't care which faucet utxo we use here so just pass lovelace 0 to grab
-  -- any present utxo
   utxo <- findFaucetUTxO node 0
   pparams <- queryProtocolParameters networkId nodeSocket QueryTip
+  let collateralTxIns = mempty
   let output =
         mkTxOutAutoBalance
           pparams
           atAddress
-          mempty
+          val
           datum
           ReferenceScriptNone
   buildTransaction
@@ -167,8 +167,7 @@ createOutputAtAddress node@RunningNode{networkId, nodeSocket} atAddress datum =
     collateralTxIns
     [output]
     >>= \case
-      Left e ->
-        throwErrorAsException e
+      Left e -> throwErrorAsException e
       Right body -> do
         let tx = makeSignedTransaction [makeShelleyKeyWitness body (WitnessPaymentKey faucetSk)] body
         submitTransaction networkId nodeSocket tx
@@ -177,8 +176,6 @@ createOutputAtAddress node@RunningNode{networkId, nodeSocket} atAddress datum =
           Nothing -> failure $ "Could not find script output: " <> decodeUtf8 (encodePretty newUtxo)
           Just u -> pure u
  where
-  collateralTxIns = mempty
-
   changeAddress = mkVkAddress networkId
 
 -- | Build and sign tx and return the calculated fee.