Integration test cluster for Byron;Shelley #1924

Anviking · 2020-07-17T23:37:31Z

Issue Number

#1920

Overview

Add support for Byron;Shelley integration test cluster
Adjusted configs to work with the new version of cardano-node
withCluster now takes three actions:
- onByron
- onFork (when in shelley. We can send money to the shelley faucets here)
- onClusterStart (when the stake pools have fully started)
CLI faucets are now byron facuets, and are generated automatically. This is both easier and more elegant.
The underlying logic of withCluster was kept somewhat flexible for easier experimentation.
Todo: For the Shelley integration tests to work, we need to send funds to the shelley faucet addresses, as initialFunds doesn't work when entering shelley through a hard-fork.
Todo clean up faucet config/code, and make sure special wallets are included

Comments

$ stack test cardano-wallet-shelley:integration

Anviking · 2020-07-21T20:33:52Z

@rvl @KtorZ I have rebased this, and the integration tests are starting, and some passing.

I'll ping/tag for review when the the final todo is done, unless you think there will be merge conflicts and we should merge this.

- Adjusted configs to work with the new version of cardano-node - withCluster now takes three actions: - onByron - onFork (when in shelley) - onClusterStart (when the stake pools have fully started) - CLI faucets are now byron facuets, and are generated automatically. This is both easier and more elegant. - The underlying logic of withCluster was kept abstract for easier experimentation. Contains a few squashed commits from Rodney.

lib/shelley/src/Cardano/Wallet/Shelley/Launch.hs

Anviking · 2020-07-22T15:53:22Z

I'll want to squash appropriately at some point, but not yet.

Anviking · 2020-07-22T16:22:22Z

lib/shelley/test/integration/Main.hs

+    afterFork dir _ = do
+        let encodeAddr = T.unpack . encodeAddress @'Mainnet
+        let addresses = map (first encodeAddr) shelleyIntegrationTestFunds
+        sendFaucetFundsTo stdoutTextTracer dir addresses


The setup is sprinkled across integration/Main.hs, Faucet.hs, and Launch.hs, which feels a bit weird. Maybe we can resolve that sometime.

Anviking · 2020-07-22T16:29:27Z

lib/shelley/test/data/cardano-node-shelley/shelley-genesis.yaml

+# b) the `systemStart` is hard-coded here to please the parser but is replaced dynamically
+#    by the same code generating the final node configuration for integration.
+#
+# c) initialFunds are defined as a list of singleton, and not as a key-value map like in the


Hm.🤔 We don't really control the order when we fill the shelley faucets now. Maybe this might cause some failures @KtorZ .

Maybe we have to make sure to always keep funds for a sequential wallet in a single tx (but we may have as many as 200 addresses to fill for a single wallet).

This will be a problem for big wallets only, which might explain why the scenario is currently failing. I thought initial funds in the shelley genesis were ignored though?

I thought initial funds in the shelley genesis were ignored though?

Yes.

(I came to think about it when reading the outdated documentation comment. Now updated.)

…stant And in particular, fix the inconsistency between fixture wallets and empty wallets that have different passphrases!

indeed, for the size estimation, we generate dummy change outputs that necessary are shelley addresses. Yet, Byron addresses have different sizes, and in particular, Byron are longer than shelley addresses (Icarus are however shorter). So, we apply a correction after the whole size calculation for each change output because generating dummy Byron addresses would be a real pain in the a**

This has been heavily reliable and is actually duplicating logic compared to the API. So instead, we query directly the API for fee estimation within the scenario and rely on a single implementation of fee estimation. This commits break every test in Shelley that are using the fee estimator, they'll be updated in subsequent commits

… calls I also removed some scenarios that were completely redundant with others (testing that the fee estimation endpoint works correctly is intrinsically done in other scenarios now that are calling it for pretty much every transactions).

The lifting was not really done wrong. We shouldn't return 'Nothing' if one of the given bound is 'Nothing'.

rvl · 2020-07-23T07:52:15Z

I'm going to merge into #1909 and we can continue there...

1909: Cardano.Api -> Cardano.Api.Typed and HardForkBlock r=KtorZ a=rvl ### Issue Number Relates to #1870. ### Overview - Updates cardano-node to latest master version. - Updates libraries to version from cardano-node master. - Cardano.Api → Cardano.Api.Typed changes. - Change `ShelleyBlock` → `CardanoBlock sc` in "Compatibility" module - Change `ShelleyBlock` → `CardanoBlock sc` in network module - Add in byron implementations of conversion functions for two-era mode. - Add CLI support for two-era mode. - Use LocalStateQuery to get the history Interpreter from the node. - Update shelley transaction layer to use new extended keys support in Cardano.Api.Typed. ### Comments - [x] Fix all the type errors in packages other than cardano-wallet-shelley. - [x] Transaction layer works during shelley era - [ ] Transaction layer works during byron era - [x] Manual testing against [mainnet_candidate](https://hydra.iohk.io/job/Cardano/iohk-nix/cardano-deployment/latest/download/1/index.html) which is in byron era -- can sync and restore empty wallet - [x] Manual testing against [shelley_qa](https://hydra.iohk.io/job/Cardano/iohk-nix/cardano-deployment/latest/download/1/index.html) which is in shelley era -- can sync and restore empty wallet - [ ] Unit tests passing - [x] Merge integration test cluster setup from #1924, and fix. Co-authored-by: Rodney Lorrimar <[email protected]> Co-authored-by: Johannes Lund <[email protected]>

Anviking self-assigned this Jul 17, 2020

rvl mentioned this pull request Jul 18, 2020

Hard fork integration tests #1920

Closed

Anviking force-pushed the anviking/1920/fork-cluster branch 6 times, most recently from cbba7eb to 6fb19d6 Compare July 19, 2020 04:27

Anviking mentioned this pull request Jul 19, 2020

Actually integrate with HFC (byron-shelley) node #1870

Closed

9 tasks

Anviking force-pushed the anviking/1920/fork-cluster branch from 6be9ba9 to 1b73bf7 Compare July 19, 2020 13:24

rvl changed the base branch from master to rvl/1870/bump-deps July 21, 2020 02:00

rvl mentioned this pull request Jul 21, 2020

Cardano.Api -> Cardano.Api.Typed and HardForkBlock #1909

Merged

7 tasks

Anviking mentioned this pull request Jul 21, 2020

cardano-node: 1.15.1 -> 1.16.x (master branch) #1925

Closed

Anviking force-pushed the anviking/1920/fork-cluster branch from ff92299 to df89bbb Compare July 21, 2020 11:36

KtorZ force-pushed the rvl/1870/bump-deps branch 2 times, most recently from 0add156 to e91d4b5 Compare July 21, 2020 17:05

Anviking force-pushed the anviking/1920/fork-cluster branch 3 times, most recently from 49d1f94 to d6d15db Compare July 21, 2020 20:08

Anviking changed the title ~~WIP: Integration test cluster for Byron;Shelley~~ Integration test cluster for Byron;Shelley Jul 21, 2020

Anviking force-pushed the anviking/1920/fork-cluster branch from 00eb494 to 6ffe092 Compare July 22, 2020 09:38

KtorZ force-pushed the rvl/1870/bump-deps branch from 59535f5 to 1db42f9 Compare July 22, 2020 12:07

Anviking added 4 commits July 22, 2020 14:16

Print cartouche when waiting for hard-fork to finish

6ab78b7

Fixup incorrect docs

0d5a311

wip: add working sendFaucetFundsTo

fff475a

KtorZ force-pushed the anviking/1920/fork-cluster branch from 6ffe092 to fff475a Compare July 22, 2020 12:17

KtorZ reviewed Jul 22, 2020

View reviewed changes

lib/shelley/src/Cardano/Wallet/Shelley/Launch.hs Outdated Show resolved Hide resolved

KtorZ reviewed Jul 22, 2020

View reviewed changes

lib/shelley/src/Cardano/Wallet/Shelley/Launch.hs Show resolved Hide resolved

Re-add call to prepareKeyRegistration, fix mistake in addr

dd0514e

Anviking commented Jul 22, 2020

View reviewed changes

Anviking and others added 7 commits July 22, 2020 20:48

Adjust documentation in shelley-genesis

d93ff57

replace hard-coded occurences of 'Secure Passphrase' with a named con…

baf60fe

…stant And in particular, fix the inconsistency between fixture wallets and empty wallets that have different passphrases!

fix 'slotRangeFromTimeRange' time interpreter query

fc44e9e

The lifting was not really done wrong. We shouldn't return 'Nothing' if one of the given bound is 'Nothing'.

Use genesis block parameters for waitForHardFork in tests

afd1e73

KtorZ force-pushed the anviking/1920/fork-cluster branch from 0b645dc to afd1e73 Compare July 23, 2020 07:26

rvl added this to the (ADP-356) Integration with Hard Fork Combinator milestone Jul 23, 2020

rvl added ADDING FEATURE Mark a PR as adding a new feature, for auto-generated CHANGELOG IMPROVEMENT Mark a PR as an improvement, for auto-generated CHANGELOG and removed ADDING FEATURE Mark a PR as adding a new feature, for auto-generated CHANGELOG labels Jul 23, 2020

rvl marked this pull request as ready for review July 23, 2020 07:51

rvl merged commit f8399ea into rvl/1870/bump-deps Jul 23, 2020

rvl deleted the anviking/1920/fork-cluster branch July 23, 2020 07:52

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Integration test cluster for Byron;Shelley #1924

Integration test cluster for Byron;Shelley #1924

Anviking commented Jul 17, 2020 •

edited

Loading

Anviking commented Jul 21, 2020 •

edited

Loading

Anviking commented Jul 22, 2020

Anviking Jul 22, 2020

Anviking Jul 22, 2020 •

edited

Loading

KtorZ Jul 22, 2020

Anviking Jul 22, 2020

rvl commented Jul 23, 2020

Integration test cluster for Byron;Shelley #1924

Integration test cluster for Byron;Shelley #1924

Conversation

Anviking commented Jul 17, 2020 • edited Loading

Issue Number

Overview

Comments

Anviking commented Jul 21, 2020 • edited Loading

Anviking commented Jul 22, 2020

Anviking Jul 22, 2020

Choose a reason for hiding this comment

Anviking Jul 22, 2020 • edited Loading

Choose a reason for hiding this comment

KtorZ Jul 22, 2020

Choose a reason for hiding this comment

Anviking Jul 22, 2020

Choose a reason for hiding this comment

rvl commented Jul 23, 2020

Anviking commented Jul 17, 2020 •

edited

Loading

Anviking commented Jul 21, 2020 •

edited

Loading

Anviking Jul 22, 2020 •

edited

Loading