Integration test coverage for migration of freeriders #2658

jonathanknowles · 2021-05-19T09:42:17Z

Issue Number

ADP-840

Overview

This PR adds integration test coverage for migrations of wallets with freerider UTxO entries.

In particular, we add two test cases:

CREATE_MIGRATION_PLAN_07
This test verifies that we can create a complete migration plan for a wallet with a large number of freerider UTxO entries, but with just enough non-freerider entries to enable the entire UTxO set to be migrated.
CREATE_MIGRATION_PLAN_08
This test verifies that we can create a partial migration plan for a wallet with a large number of freerider UTxO entries, but with not quite enough non-freerider entries to enable the entire UTxO set to be migrated.

In addition, this PR:

Adds a synchronization lock around the mintSeaHorseAssets operation, as it is not safe to be called by more than one thread concurrently.

This test verifies that we can create a complete migration plan for a wallet with a large number of freerider UTxO entries, but with just enough non-freerider entries to enable the entire UTxO set to be migrated.

This test verifies that we can create a partial migration plan for a wallet with a large number of freerider UTxO entries, but with not quite enough non-freerider entries to enable the entire UTxO set to be migrated.

This commit just clarifies the comment and constants associated with the minting of assets in test `SHELLEY_CREATE_MIGRATION_PLAN_04`.

These tests are slow to run. If one of these tests fails, it's very unlikely to succeed when run a second time.

We do this by 1. Avoid multiple calls to waitForTxImmutability 2. Use batchSize to minting using fewer txs Now we have SHELLEY_CREATE_MIGRATION_PLAN_07 - Can create a complete migration plan for a wallet with a large number of freerider UTxO entries, but with just enough non-freerider entries to enable the entire UTxO set to be migrated. Finished in 63.8423 seconds Instead of SHELLEY_CREATE_MIGRATION_PLAN_07 - Can create a complete migration plan for a wallet with a large number of freerider UTxO entries, but with just enough non-freerider entries to enable the entire UTxO set to be migrated. Finished in 149.6189 seconds And similar for 08.

Anviking · 2021-05-20T12:03:24Z

lib/core-integration/src/Test/Integration/Scenario/API/Shelley/Migrations.hs

+                , expectField (#balanceSelected . #ada . #getQuantity)
+                    (`shouldBe` 257_812_500)
+                , expectField (#balanceLeftover . #ada . #getQuantity)
+                    (`shouldBe`  29_687_500)
+                , expectField (#balanceSelected . #assets . #getApiT)
+                    ((.> 0)
+                        . Set.size
+                        . TokenMap.getAssets)
+                , expectField (#balanceLeftover . #assets . #getApiT)
+                    ((.> 0)
+                        . Set.size
+                        . TokenMap.getAssets)


jonathanknowles · 2021-05-20T12:08:41Z

bors r+

2658: Integration test coverage for migration of freeriders r=jonathanknowles a=jonathanknowles # Issue Number ADP-840 # Overview This PR adds integration test coverage for migrations of wallets with freerider UTxO entries. In particular, we add two test cases: - [x] `CREATE_MIGRATION_PLAN_07` This test verifies that we can create a **_complete_** migration plan for a wallet with a large number of freerider UTxO entries, but with **_just enough_** non-freerider entries to enable the entire UTxO set to be migrated. - [x] `CREATE_MIGRATION_PLAN_08` This test verifies that we can create a **_partial_** migration plan for a wallet with a large number of freerider UTxO entries, but with **_not quite enough_** non-freerider entries to enable the entire UTxO set to be migrated. In addition, this PR: - [x] Adds a synchronization lock around the `mintSeaHorseAssets` operation, as it is not safe to be called by more than one thread concurrently. Co-authored-by: Jonathan Knowles <[email protected]> Co-authored-by: Johannes Lund <[email protected]>

iohk-bors · 2021-05-20T12:53:29Z

Build failed:

buildkite/cardano-wallet


cardano-wallet             &gt;   src/Test/Integration/Scenario/CLI/Shelley/Transactions.hs:387:59:
--
&nbsp; | cardano-wallet             &gt;   1) CLI Specifications, SHELLEY_CLI_TRANSACTIONS, TRANS_ESTIMATE_09 - Invalid amount, string with wildcards
&nbsp; | cardano-wallet             &gt;        uncaught exception: IOException of type ResourceVanished
&nbsp; | cardano-wallet             &gt;        fd:136: hFlush: resource vanished (Broken pipe)
&nbsp; | cardano-wallet             &gt;
&nbsp; | cardano-wallet             &gt;   To rerun use: --match "/CLI Specifications/SHELLEY_CLI_TRANSACTIONS/TRANS_ESTIMATE_09 - Invalid amount/string with wildcards/"
&nbsp; | cardano-wallet             &gt;
&nbsp; | cardano-wallet             &gt;   src/Test/Integration/Scenario/CLI/Shelley/HWWallets.hs:152:26:
&nbsp; | cardano-wallet             &gt;   2) CLI Specifications, SHELLEY_CLI_HW_WALLETS, HW_WALLETS_01x - Restoration from account public key preserves funds
&nbsp; | cardano-wallet             &gt;        While verifying ApiWallet {id = ApiT {getApiT = WalletId {getWalletId = 89ee471d2abc67212bac4a967272cfd8bf5a3ac5}}, addressPoolGap = ApiT {getApiT = AddressPoolGap {getAddressPoolGap = 20}}, balance = ApiWalletBalance {available = Quantity {getQuantity = 0}, total = Quantity {getQuantity = 0}, reward = Quantity {getQuantity = 0}}, assets = ApiWalletAssetsBalance {available = ApiT {getApiT = TokenMap (fromList [])}, total = ApiT {getApiT = TokenMap (fromList [])}}, delegation = ApiWalletDelegation {active = ApiWalletDelegationNext {status = NotDelegating, target = Nothing, changesAt = Nothing}, next = []}, name = ApiT {getApiT = WalletName {getWalletName = "Wallet from pub key"}}, passphrase = Nothing, state = ApiT {getApiT = Syncing (Quantity {getQuantity = Percentage {getPercentage = 723 % 1250}})}, tip = ApiBlockReference {absoluteSlotNumber = ApiT {getApiT = SlotNo 4163}, slotId = ApiSlotId {epochNumber = ApiT {getApiT = EpochNo {unEpochNo = 83}}, slotNumber = ApiT {getApiT = SlotInEpoch {unSlotInEpoch = 13}}}, time = 2021-05-20 12:39:44.6 UTC, block = ApiBlockInfo {height = Quantity {getQuantity = 2000}}}}
&nbsp; | cardano-wallet             &gt;        Waited longer than 90s to resolve action: "Wallet balance is as expected on wallet from pubKey".

Anviking · 2021-05-20T12:55:44Z

bors r+

2658: Integration test coverage for migration of freeriders r=Anviking a=jonathanknowles # Issue Number ADP-840 # Overview This PR adds integration test coverage for migrations of wallets with freerider UTxO entries. In particular, we add two test cases: - [x] `CREATE_MIGRATION_PLAN_07` This test verifies that we can create a **_complete_** migration plan for a wallet with a large number of freerider UTxO entries, but with **_just enough_** non-freerider entries to enable the entire UTxO set to be migrated. - [x] `CREATE_MIGRATION_PLAN_08` This test verifies that we can create a **_partial_** migration plan for a wallet with a large number of freerider UTxO entries, but with **_not quite enough_** non-freerider entries to enable the entire UTxO set to be migrated. In addition, this PR: - [x] Adds a synchronization lock around the `mintSeaHorseAssets` operation, as it is not safe to be called by more than one thread concurrently. Co-authored-by: Jonathan Knowles <[email protected]> Co-authored-by: Johannes Lund <[email protected]>

iohk-bors · 2021-05-20T13:41:05Z

Build failed:

ci/hydra-build:required

    JSON encoding of ApiAsset
      produces the same JSON as is found in test/data/Cardano/Wallet/Api/ApiAsset.json
  Textual encoding
building of '/nix/store/wqr1dp2mgjpiyzr1i4pfl1nb7pd0wm0g-cardano-wallet-core-test-unit-2021.4.28-check' timed out after 900 seconds of silence

#2472

Anviking · 2021-05-20T13:44:27Z

bors r+

2658: Integration test coverage for migration of freeriders r=Anviking a=jonathanknowles # Issue Number ADP-840 # Overview This PR adds integration test coverage for migrations of wallets with freerider UTxO entries. In particular, we add two test cases: - [x] `CREATE_MIGRATION_PLAN_07` This test verifies that we can create a **_complete_** migration plan for a wallet with a large number of freerider UTxO entries, but with **_just enough_** non-freerider entries to enable the entire UTxO set to be migrated. - [x] `CREATE_MIGRATION_PLAN_08` This test verifies that we can create a **_partial_** migration plan for a wallet with a large number of freerider UTxO entries, but with **_not quite enough_** non-freerider entries to enable the entire UTxO set to be migrated. In addition, this PR: - [x] Adds a synchronization lock around the `mintSeaHorseAssets` operation, as it is not safe to be called by more than one thread concurrently. Co-authored-by: Jonathan Knowles <[email protected]> Co-authored-by: Johannes Lund <[email protected]>

iohk-bors · 2021-05-20T14:35:21Z

Build failed:

ci/hydra-build:required

Test failure: (log)

     JSON encoding of ApiMaintenanceAction
     allows to encode values with aeson and read them back
       +++ OK, passed 100 tests.
   JSON encoding of ApiMaintenanceAction
     produces the same JSON as is found in test/data/Cardano/Wallet/Api/ApiMaintenanceAction.json
   JSON encoding of ApiMaintenanceActionPostData
     allows to encode values with aeson and read them back
       +++ OK, passed 100 tests.
   JSON encoding of ApiMaintenanceActionPostData
     produces the same JSON as is found in test/data/Cardano/Wallet/Api/ApiMaintenanceActionPostData.json
   JSON encoding of ApiAsset
     allows to encode values with aeson and read them back
       +++ OK, passed 100 tests.
   JSON encoding of ApiAsset
     produces the same JSON as is found in test/data/Cardano/Wallet/Api/ApiAsset.json
 Textual encoding
building of '/nix/store/3ix888hw1hg8abvpbxplpq7fzj2g60mq-cardano-wallet-core-test-unit-2021.4.28-check' timed out after 900 seconds of silence

#2472

jonathanknowles · 2021-05-20T14:36:53Z

bors r+

2658: Integration test coverage for migration of freeriders r=jonathanknowles a=jonathanknowles # Issue Number ADP-840 # Overview This PR adds integration test coverage for migrations of wallets with freerider UTxO entries. In particular, we add two test cases: - [x] `CREATE_MIGRATION_PLAN_07` This test verifies that we can create a **_complete_** migration plan for a wallet with a large number of freerider UTxO entries, but with **_just enough_** non-freerider entries to enable the entire UTxO set to be migrated. - [x] `CREATE_MIGRATION_PLAN_08` This test verifies that we can create a **_partial_** migration plan for a wallet with a large number of freerider UTxO entries, but with **_not quite enough_** non-freerider entries to enable the entire UTxO set to be migrated. In addition, this PR: - [x] Adds a synchronization lock around the `mintSeaHorseAssets` operation, as it is not safe to be called by more than one thread concurrently. Co-authored-by: Jonathan Knowles <[email protected]> Co-authored-by: Johannes Lund <[email protected]>

iohk-bors · 2021-05-20T15:38:18Z

Build failed:

ci/hydra-build:required

Failure: (log)

    JSON encoding of ApiTxMetadata
    allows to encode values with aeson and read them back
      +++ OK, passed 100 tests.
  JSON encoding of ApiTxMetadata
    produces the same JSON as is found in test/data/Cardano/Wallet/Api/ApiTxMetadata.json
  JSON encoding of ApiMaintenanceAction
    allows to encode values with aeson and read them back
      +++ OK, passed 100 tests.
  JSON encoding of ApiMaintenanceAction
    produces the same JSON as is found in test/data/Cardano/Wallet/Api/ApiMaintenanceAction.json
  JSON encoding of ApiMaintenanceActionPostData
    allows to encode values with aeson and read them back
      +++ OK, passed 100 tests.
  JSON encoding of ApiMaintenanceActionPostData
    produces the same JSON as is found in test/data/Cardano/Wallet/Api/ApiMaintenanceActionPostData.json
  JSON encoding of ApiAsset
    allows to encode values with aeson and read them back
      +++ OK, passed 100 tests.
  JSON encoding of ApiAsset
    produces the same JSON as is found in test/data/Cardano/Wallet/Api/ApiAsset.json
Textual encoding
building of '/nix/store/hn7mmff6sw5fgznnga1wqbhawzsvm67n-cardano-wallet-core-test-unit-2021.4.28-check' timed out after 900 seconds of silence

#2472

jonathanknowles · 2021-05-20T15:42:02Z

bors r+

2658: Integration test coverage for migration of freeriders r=jonathanknowles a=jonathanknowles # Issue Number ADP-840 # Overview This PR adds integration test coverage for migrations of wallets with freerider UTxO entries. In particular, we add two test cases: - [x] `CREATE_MIGRATION_PLAN_07` This test verifies that we can create a **_complete_** migration plan for a wallet with a large number of freerider UTxO entries, but with **_just enough_** non-freerider entries to enable the entire UTxO set to be migrated. - [x] `CREATE_MIGRATION_PLAN_08` This test verifies that we can create a **_partial_** migration plan for a wallet with a large number of freerider UTxO entries, but with **_not quite enough_** non-freerider entries to enable the entire UTxO set to be migrated. In addition, this PR: - [x] Adds a synchronization lock around the `mintSeaHorseAssets` operation, as it is not safe to be called by more than one thread concurrently. Co-authored-by: Jonathan Knowles <[email protected]> Co-authored-by: Johannes Lund <[email protected]>

iohk-bors · 2021-05-20T16:35:09Z

Build failed:

ci/hydra-build:required

Failure: (https://hydra.iohk.io/build/6431843/nixlog/1/tail)

    JSON encoding of ApiTxMetadata
      allows to encode values with aeson and read them back
        +++ OK, passed 100 tests.
    JSON encoding of ApiTxMetadata
      produces the same JSON as is found in test/data/Cardano/Wallet/Api/ApiTxMetadata.json
    JSON encoding of ApiMaintenanceAction
      allows to encode values with aeson and read them back
        +++ OK, passed 100 tests.
    JSON encoding of ApiMaintenanceAction
      produces the same JSON as is found in test/data/Cardano/Wallet/Api/ApiMaintenanceAction.json
    JSON encoding of ApiMaintenanceActionPostData
      allows to encode values with aeson and read them back
        +++ OK, passed 100 tests.
    JSON encoding of ApiMaintenanceActionPostData
      produces the same JSON as is found in test/data/Cardano/Wallet/Api/ApiMaintenanceActionPostData.json
    JSON encoding of ApiAsset
      allows to encode values with aeson and read them back
        +++ OK, passed 100 tests.
    JSON encoding of ApiAsset
      produces the same JSON as is found in test/data/Cardano/Wallet/Api/ApiAsset.json
  Textual encoding
building of '/nix/store/ihz8qn4fvqk0nb1glh7ymiivaz0cg15z-cardano-wallet-core-test-unit-2021.4.28-check' timed out after 900 seconds of silence

#2472

Anviking · 2021-05-20T17:44:33Z

bors r+

2658: Integration test coverage for migration of freeriders r=Anviking a=jonathanknowles # Issue Number ADP-840 # Overview This PR adds integration test coverage for migrations of wallets with freerider UTxO entries. In particular, we add two test cases: - [x] `CREATE_MIGRATION_PLAN_07` This test verifies that we can create a **_complete_** migration plan for a wallet with a large number of freerider UTxO entries, but with **_just enough_** non-freerider entries to enable the entire UTxO set to be migrated. - [x] `CREATE_MIGRATION_PLAN_08` This test verifies that we can create a **_partial_** migration plan for a wallet with a large number of freerider UTxO entries, but with **_not quite enough_** non-freerider entries to enable the entire UTxO set to be migrated. In addition, this PR: - [x] Adds a synchronization lock around the `mintSeaHorseAssets` operation, as it is not safe to be called by more than one thread concurrently. Co-authored-by: Jonathan Knowles <[email protected]> Co-authored-by: Johannes Lund <[email protected]>

iohk-bors · 2021-05-20T20:01:08Z

Build failed:

ci/hydra-build:required

Failure: (https://hydra.iohk.io/build/6432955/nixlog/1/tail)

  80) CLI Specifications, SHELLEY_CLI_WALLETS, WALLETS_UTXO_02 - Utxo statistics works properly
       Waited longer than 90s to resolve action: "Wallet balance is as expected".
       expected: Quantity {getQuantity = 1562000000}
        but got: Quantity {getQuantity = 0}

  To rerun use: --match "/CLI Specifications/SHELLEY_CLI_WALLETS/WALLETS_UTXO_02 - Utxo statistics works properly/"

  src/Test/Integration/Scenario/CLI/Shelley/HWWallets.hs:129:18:
  81) CLI Specifications, SHELLEY_CLI_HW_WALLETS, HW_WALLETS_01x - Restoration from account public key preserves funds
       Waited longer than 90s to resolve action: "Wallet balance is as expected".
       expected: Quantity {getQuantity = 1000000}
        but got: Quantity {getQuantity = 0}

  To rerun use: --match "/CLI Specifications/SHELLEY_CLI_HW_WALLETS/HW_WALLETS_01x - Restoration from account public key preserves funds/"

#2428

jonathanknowles · 2021-05-21T00:46:01Z

bors r+

iohk-bors · 2021-05-21T01:28:39Z

Build succeeded:

jonathanknowles force-pushed the jonathanknowles/migration-freerider-integration-tests branch from 31b8882 to 1264f43 Compare May 20, 2021 06:51

jonathanknowles self-assigned this May 20, 2021

jonathanknowles force-pushed the jonathanknowles/migration-freerider-integration-tests branch from 1264f43 to cb06a2c Compare May 20, 2021 06:53

jonathanknowles requested a review from Anviking May 20, 2021 08:38

jonathanknowles marked this pull request as ready for review May 20, 2021 08:38

Anviking force-pushed the jonathanknowles/migration-freerider-integration-tests branch from e364763 to 414e5b0 Compare May 20, 2021 09:14

jonathanknowles and others added 7 commits May 20, 2021 09:55

Prevent concurrent minting of seahorse assets.

fdf18df

Add test SHELLEY_CREATE_MIGRATION_PLAN_07.

e83c0f7

This test verifies that we can create a complete migration plan for a wallet with a large number of freerider UTxO entries, but with just enough non-freerider entries to enable the entire UTxO set to be migrated.

Add test SHELLEY_CREATE_MIGRATION_PLAN_08.

77db4e7

This test verifies that we can create a partial migration plan for a wallet with a large number of freerider UTxO entries, but with not quite enough non-freerider entries to enable the entire UTxO set to be migrated.

Clarify asset minting in SHELLEY_CREATE_MIGRATION_PLAN_04.

91006c7

This commit just clarifies the comment and constants associated with the minting of assets in test `SHELLEY_CREATE_MIGRATION_PLAN_04`.

Don't allow freerider integration tests to be retried.

1d2d6cb

These tests are slow to run. If one of these tests fails, it's very unlikely to succeed when run a second time.

Simplify minting in SHELLEY_CREATE_MIGRATION_PLAN_{07,08}.

29b7c46

jonathanknowles force-pushed the jonathanknowles/migration-freerider-integration-tests branch from 414e5b0 to 29b7c46 Compare May 20, 2021 10:14

Anviking approved these changes May 20, 2021

View reviewed changes

iohk-bors bot merged commit 1bf7164 into master May 21, 2021

iohk-bors bot deleted the jonathanknowles/migration-freerider-integration-tests branch May 21, 2021 01:28

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Integration test coverage for migration of freeriders #2658

Integration test coverage for migration of freeriders #2658

jonathanknowles commented May 19, 2021 •

edited

Loading

Anviking May 20, 2021

jonathanknowles commented May 20, 2021

iohk-bors bot commented May 20, 2021 •

edited by Anviking

Loading

Anviking commented May 20, 2021

iohk-bors bot commented May 20, 2021 •

edited by Anviking

Loading

Anviking commented May 20, 2021

iohk-bors bot commented May 20, 2021 •

edited by jonathanknowles

Loading

jonathanknowles commented May 20, 2021

iohk-bors bot commented May 20, 2021 •

edited by jonathanknowles

Loading

jonathanknowles commented May 20, 2021

iohk-bors bot commented May 20, 2021 •

edited by jonathanknowles

Loading

Anviking commented May 20, 2021

iohk-bors bot commented May 20, 2021 •

edited by jonathanknowles

Loading

jonathanknowles commented May 21, 2021

iohk-bors bot commented May 21, 2021

Integration test coverage for migration of freeriders #2658

Integration test coverage for migration of freeriders #2658

Conversation

jonathanknowles commented May 19, 2021 • edited Loading

Issue Number

Overview

Anviking May 20, 2021

Choose a reason for hiding this comment

jonathanknowles commented May 20, 2021

iohk-bors bot commented May 20, 2021 • edited by Anviking Loading

Anviking commented May 20, 2021

iohk-bors bot commented May 20, 2021 • edited by Anviking Loading

Anviking commented May 20, 2021

iohk-bors bot commented May 20, 2021 • edited by jonathanknowles Loading

jonathanknowles commented May 20, 2021

iohk-bors bot commented May 20, 2021 • edited by jonathanknowles Loading

jonathanknowles commented May 20, 2021

iohk-bors bot commented May 20, 2021 • edited by jonathanknowles Loading

Anviking commented May 20, 2021

iohk-bors bot commented May 20, 2021 • edited by jonathanknowles Loading

jonathanknowles commented May 21, 2021

iohk-bors bot commented May 21, 2021

jonathanknowles commented May 19, 2021 •

edited

Loading

iohk-bors bot commented May 20, 2021 •

edited by Anviking

Loading

iohk-bors bot commented May 20, 2021 •

edited by Anviking

Loading

iohk-bors bot commented May 20, 2021 •

edited by jonathanknowles

Loading

iohk-bors bot commented May 20, 2021 •

edited by jonathanknowles

Loading

iohk-bors bot commented May 20, 2021 •

edited by jonathanknowles

Loading

iohk-bors bot commented May 20, 2021 •

edited by jonathanknowles

Loading