Enable parallel integration tests in hydra

[Anviking]

Issue Number

ADP-466

Overview

• Pass in -j 8
• Only run CLI tests in parallel if CI env var isn't set
• Clean up some debug printing in tests

Comments

[Anviking]

bors try

[iohk-bors]

try

Build failed:

• buildkite/cardano-wallet

Failures:
--
  |  
  | src/Test/Integration/Scenario/API/Shelley/Transactions.hs:641:10:
  | 1) API Specifications, SHELLEY_TRANSACTIONS, TRANS_TTL_01 - Pending transaction expiry
  | predicate failed on: SlotNo 86
  | expected expiry: SlotNo 37553
  | actual expiry: SlotNo 37639
  |  
  | To rerun use: --match "/API Specifications/SHELLEY_TRANSACTIONS/TRANS_TTL_01 - Pending transaction expiry/"

#2295

[Anviking]

bors try

[iohk-bors]

try

Build failed:

• buildkite/cardano-wallet

#expected

--
  | • Variable not in scope: counterexample :: String -> m0 () -> IO a0
  | • Perhaps you want to add ‘counterexample’ to the import list
  | in the import of ‘Test.Integration.Framework.DSL’
  | (test/integration/Test/Integration/Jormungandr/Scenario/CLI/Server.hs:(49,1)-(55,5)).
  | \|
  | 171 \|                 counterexample logged $
  | \|                 ^^^^^^^^^^^^^^

[Anviking]

bors try

[iohk-bors]

try

Build failed:

• buildkite/cardano-wallet

I think I had included an incorrect fix for TRANS_CREATE_03 here, so
#expected

src/Test/Integration/Scenario/API/Shelley/Transactions.hs:394:5:
--
  | 1) API Specifications, SHELLEY_TRANSACTIONS, TRANS_CREATE_03 - 0 balance after transaction
  | uncaught exception: RequestException
  | DecodeFailure "Error in $: parsing Cardano.Wallet.Api.Types.ApiFee(ApiFee) failed, key \"estimated_min\" not found: Response {responseStatus = Status {statusCode = 403, statusMessage = \"Forbidden\"}, responseVersion = HTTP/1.1, responseHeaders = [(\"Transfer-Encoding\",\"chunked\"),(\"Date\",\"Thu, 05 Nov 2020 14:43:44 GMT\"),(\"Server\",\"Warp/3.3.5\")], responseBody = \"{\\\"code\\\":\\\"not_enough_money\\\",\\\"message\\\":\\\"I can't process this payment because there's not enough UTxO available in the wallet. The total UTxO sums up to 0 Lovelace, but I need 1000000 Lovelace (excluding fee amount) in order to proceed  with the payment.\\\"}\", responseCookieJar = CJ {expose = []}, responseClose' = ResponseClose}"
  |  
  | To rerun use: --match "/API Specifications/SHELLEY_TRANSACTIONS/TRANS_CREATE_03 - 0 balance after transaction/"

[Anviking]

bors try

[iohk-bors]

try

Merge conflict.

[Anviking]

bors try

[iohk-bors]

try

Build failed:

• ci/hydra-build:required

[Anviking]

bors try

[iohk-bors]

try

Build failed:

• ci/hydra-build:required

[Anviking]

bors try

[iohk-bors]

try

Build failed:

• ci/hydra-build:required

Cardano.Wallet.DB.MVar
    ...
    parallel puts replace values in _any_ order
      Checkpoint
        +++ OK, passed 100 tests (69% conflicting db entries).
      Wallet Metadata
        +++ OK, passed 100 tests (69% conflicting db entries).
      Tx History
        +++ OK, passed 100 tests (69% conflicting db entries).
building of '/nix/store/ksc84i49szl7y4ysnfrcwl4clyhnmh10-cardano-wallet-core-test-unit-2020.11.3-x86_64-w64-mingw32-check-x86_64-w64-mingw32' timed out after 7200 seconds of silence

#2292

[Anviking]

bors try

[iohk-bors]

Build failed:

• buildkite/cardano-wallet

src/Test/Integration/Scenario/API/Shelley/Transactions.hs:675:10:
--
  | 1) API Specifications, SHELLEY_TRANSACTIONS, TRANS_TTL_02 - Custom transaction expiry
  | predicate failed on: SlotNo 62
  | expected expiry: SlotNo 1835
  | actual expiry: SlotNo 1897
  |  
  | To rerun use: --match "/API Specifications/SHELLEY_TRANSACTIONS/TRANS_TTL_02 - Custom transaction expiry/"

#2295

[Anviking]

bors r+

[iohk-bors]

Build failed:

• ci/hydra-build:required

    getTx properties
      can read after putting tx history for valid tx id
        +++ OK, passed 100 tests.
      cannot read after putting tx history for invalid tx id
        +++ OK, passed 100 tests.
      cannot read after putting tx history for invalid wallet id
        +++ OK, passed 100 tests; 10 discarded.
building of '/nix/store/2irxs3bmzsfn6929dsnn47niacgcsd6b-cardano-wallet-core-test-unit-2020.11.3-check' timed out after 900 seconds of silence

Probably prop_putBeforeInit timing out, since "cannot read after putting tx history for invalid wallet id" passed.
#2292

[Anviking]

bors r+

[iohk-bors]

Build failed:

• buildkite/cardano-wallet

   parallel puts replace values in _any_ order
      Checkpoint
        +++ OK, passed 100 tests (73% conflicting db entries).
      Wallet Metadata
        +++ OK, passed 100 tests (73% conflicting db entries).
      Tx History
        +++ OK, passed 100 tests (73% conflicting db entries).
      Private Key
        +++ OK, passed 100 tests (73% conflicting db entries).
    rollback
      Can rollback to any arbitrary known checkpoint
        +++ OK, passed 100 tests.
      Correctly re-construct tx history on rollbacks
        +++ OK, passed 200 tests:
        69.0% rolling back something
        
        31.0% Outgoing tx after point: 0
        18.0% Outg

#2292

[Anviking]

bors r+

[iohk-bors]

Build failed:

• buildkite/cardano-wallet

Unit tests timed out.

    parallel puts replace values in _any_ order
      Checkpoint
        +++ OK, passed 100 tests (68% conflicting db entries).
      Wallet Metadata
        +++ OK, passed 100 tests (68% conflicting db entries).
      Tx History
        +++ OK, passed 100 tests (68% conflicting db entries).
      Private Key
        +++ OK, passed 100 tests (68% conflicting db entries).
    rollback
      Can rollback to any arbitrary known checkpoint
        +++ OK, passed 100 tests.
      Correctly re-construct tx history on rollbacks
        +++ OK, passed 400 tests:
        64.2% rolling back something
        35.8% Outgoing tx after po

#2292 (seems similar to, at least)

[Anviking]

bors r+

[iohk-bors]

Build failed:

• buildkite/cardano-wallet

Progress 6/7: cardano-wallet-2020.11.3
--
  | Timed out after 60 minutes.

    STAKE_POOLS_LIST_01 - List stake pools
OOkOk.k.
.

Ok.
Ok.
Ok.
Ok.
Ok.
Ok.O
k.
Ok.
Ok.
Ok.
Ok.
Ok.

Integration tests timed out

#2311

[Anviking]

bors r+

[iohk-bors]

Build failed:

• buildkite/cardano-wallet

src/Test/Integration/Scenario/API/Shelley/Transactions.hs:641:10:
--
  | 1) API Specifications, SHELLEY_TRANSACTIONS, TRANS_TTL_01 - Pending transaction expiry
  | predicate failed on: SlotNo 1070
  | expected expiry: SlotNo 38152
  | actual expiry: SlotNo 39222
  |  
  | To rerun use: --match "/API Specifications/SHELLEY_TRANSACTIONS/TRANS_TTL_01 - Pending transaction expiry/"
  |  
  | src/Test/Integration/Scenario/API/Shelley/Transactions.hs:675:10:
  | 2) API Specifications, SHELLEY_TRANSACTIONS, TRANS_TTL_02 - Custom transaction expiry
  | predicate failed on: SlotNo 1086
  | expected expiry: SlotNo 2362
  | actual expiry: SlotNo 3448
  |  
  | To rerun use: --match "/API Specifications/SHELLEY_TRANSACTIONS/TRANS_TTL_02 - Custom transaction expiry/"
  |  
  | src/Test/Integration/Scenario/CLI/Shelley/Wallets.hs:244:49:
  | 3) CLI Specifications, SHELLEY_CLI_WALLETS, WALLETS_CREATE_02 - Restored wallet preserves funds
  | Waited longer than 90s to resolve action: "Wallet is fully restored".
  | expected: Ready
  | but got: Syncing (Quantity {getQuantity = Percentage {getPercentage = 1977 % 2000}})
  |  
  | To rerun use: --match "/CLI Specifications/SHELLEY_CLI_WALLETS/WALLETS_CREATE_02 - Restored wallet preserves funds/"
  |  
  | src/Test/Integration/Scenario/CLI/Shelley/Wallets.hs:744:18:
  | 4) CLI Specifications, SHELLEY_CLI_WALLETS, WALLETS_UTXO_02 - Utxo statistics works properly
  | Waited longer than 90s to resolve action: "Wallet balance is as expected".
  | expected: Quantity {getQuantity = 1562000000}
  | but got: Quantity {getQuantity = 0}
  |  
  | To rerun use: --match "/CLI Specifications/SHELLEY_CLI_WALLETS/WALLETS_UTXO_02 - Utxo statistics works properly/"
  |  
  | src/Test/Integration/Scenario/CLI/Shelley/HWWallets.hs:132:18:
  | 5) CLI Specifications, SHELLEY_CLI_HW_WALLETS, HW_WALLETS_01x - Restoration from account public key preserves funds
  | Waited longer than 90s to resolve action: "Wallet balance is as expected".
  | expected: Quantity {getQuantity = 1000000}
  | but got: Quantity {getQuantity = 0}
  |  
  | To rerun use: --match "/CLI Specifications/SHELLEY_CLI_HW_WALLETS/HW_WALLETS_01x - Restoration from account public key preserves funds/"
  |  
  | Randomized with seed 1574735734
  |  
  | Finished in 1552.9214 seconds
  | 697 examples, 5 failures, 5 pending

#2295 + more

[Anviking]

bors r+

[iohk-bors]

Build failed (retrying...):

• ci/hydra-build:required

    getTx properties
      can read after putting tx history for valid tx id
        +++ OK, passed 100 tests.
      cannot read after putting tx history for invalid tx id
        +++ OK, passed 100 tests.
      cannot read after putting tx history for invalid wallet id
        +++ OK, passed 100 tests; 13 discarded.
    can't put before wallet exists
      Checkpoint
building of '/nix/store/q5csa5xaphm3yzg537lmvgzl8nz9syj9-cardano-wallet-core-test-unit-2020.11.3-x86_64-unknown-linux-musl-check-x86_64-unknown-linux-musl' timed out after 7200 seconds of silence

#2292

[iohk-bors]

Build failed (retrying...):

• buildkite/cardano-wallet

SQLite3 returned ErrorMisuse while attempting to perform bind int64.
--
  | Ok.
  | Ok.
  | Ok.
  | malloc(): invalid size (unsorted)

Last successful test ADDRESS_IMPORT_04 - I can import a used address (idempotence)
Then many Ok. ...
And finally:

SQLite3 returned ErrorMisuse while attempting to perform bind int64.
--
  | Ok.
  | Ok.
  | Ok.
  | malloc(): invalid size (unsorted)

#2350

[iohk-bors]

Build succeeded:

• buildkite/cardano-wallet
• ci/hydra-build:required