expose an executable 'shelley-test-cluster' which starts an integration test cluster with faucets #2178

KtorZ · 2020-09-24T09:43:10Z

Issue Number

Overview

9722d82
📍 expose an executable 'shelley-test-cluster' which starts an integration test cluster with faucets
6eefd47
📍 Let the shelley-test-cluster nix build be run from any directory
4c4d01e
📍 Allow setting different LoggerName when configuring trace with 'withLogging'
571e67d
📍 Add TestLog to shelley-test-cluster
e99d875
📍 Regenerate nix

Comments

~~@Anviking there's probably some clashes with what you're also doing with the logging, so happy to get the logging revision merged first and adjust that one.~~

Anviking · 2020-09-24T10:53:37Z

lib/shelley/exe/shelley-test-cluster.hs

+--
+-- - (Shelley) Contains only small coins (but greater than the minUTxOValue)
+--
+--     [ "either" , "flip" , "maple" , "shift" , "dismiss"


Nice docs, but feels like it might get out of date very easily? Also is a doc comment actually the right place for this information?

Maybe we could add a new shelley-test-cluster mnemonics, and change shelleyIntegrationTestFunds to also include a String description for each wallet?

The output of such a command would be huge. We could add some logic to protect against it, or maybe just something like grep:

E.g.

$ shelley-test-cluster query-mnemonic "small coins" shelley/special/seq/Contains only small coins (but greater than the minUTxOValue) 1 either flip maple shift dismiss bridge sweet reveal green tornado need patient wall stamp pass shelley/special/seq/Contains only small coins (but greater than the minUTxOValue) 2 reveal sweet green tornado need patient wall stamp pass either flip maple shift dismiss bridge

Nice docs, but feels like it might get out of date very easily? Also is a doc comment actually the right place for this information?

I wondered the same and actually considered the same idea you're proposing here. Although arguably:

we haven't changed those mnemonics in several months and have no reason to change them in the upcoming future.

that is for now, mostly an internal tool, so if this end up outdated, we'll most likely notice it quite quickly.

So, the size of the problem seems not large enough to spend effort on making this better 😅 ... arguably.

I mean, the same argument applies for the pools configurations and even the cluster setup (what if we add a new node tomorrow? the doc becomes outdated). All this doc describes implementation details, so by its very nature, it's fragile.

I remember having to re-regenerate and update faucet mnemonics several for Shelley, which was not fun at all. With the HFC integration we finally got it much more automated.

I think just adding a command to output all mnemonics with a description, requiring manual | grep "" piping, would have been really simple.

But sure, it's perhaps unlikely they'll change now. And perhaps still best to keep it comment, for now.

Btw, I wonder if this could be used for ADP-381 eventually.

Perhaps ultimately, we can clean-up our integration tests setup to use that executable. Spawn it in a process and go ahead with the rest of the tests. Right now the integration setup is very complicated with contexts and mvars a bit everywhere waiting for stuff.

Anviking · 2020-09-29T07:44:23Z

I can skim over again when rebased.

But trying now I get:

  cardano-cli: pokeSockAddr: path is too long
  CallStack (from HasCallStack):
    error, called at Network/Socket/Types.hsc:1077:40 in network-3.1.1.1-Izwsyk64OoDBVuRysfRa9D:Network.Socket.Types

λ> length "/private/var/folders/h_/mlzym27d5lz_fx3ph2d547700000gn/T/shelley-test-cluster-ff9e0fd9e9507b24/bft/node.socket"
110

seems 6 characters too long on macOS. Using withSystemTempDir trCluster "shelleycluster" instead of "shelley-test-cluster" seems to fix it.

lib/shelley/exe/shelley-test-cluster.hs

KtorZ · 2020-10-01T15:32:08Z

bors merge

@Anviking

2178: expose an executable 'shelley-test-cluster' which starts an integration test cluster with faucets r=KtorZ a=KtorZ # Issue Number  # Overview  - [ ] I have exposed our integration test setup in an executable. # Comments  @Anviking there's probably some clashes with what you're also doing with the logging, so happy to get the logging revision merged first and adjust that one.  Co-authored-by: KtorZ <[email protected]> Co-authored-by: Rodney Lorrimar <[email protected]> Co-authored-by: IOHK <[email protected]>

Anviking · 2020-10-01T15:43:36Z

lib/shelley/src/Cardano/Wallet/Shelley/Launch.hs

@@ -1648,6 +1715,8 @@ instance ToText ClusterLog where
        MsgGenOperatorKeyPair dir ->
            "Generating stake pool operator key pair in " <> T.pack dir
        MsgCLI args -> T.pack $ unwords ("cardano-cli":args)
+        MsgListenAddress addr ->
+            "Wallet backend server listening on " <> T.pack (show addr)


I don't think this should be added.

For cardano-wallet:integration there is a separation between TestsLog and ClusterLog, where MsgListenAddress would make sense in TestsLog, but not ClusterLog.

We could add a similar TestsLog for this executable in a follow-up PR.

Hmmm. Possibly. Although this one should really be in the application logs of the wallet.

lib/shelley/src/Cardano/Wallet/Shelley/Launch.hs

Anviking

Unless someone wants needs this urgently, can we address at least #2178 (comment) first?

Anviking · 2020-10-01T16:39:55Z

bors r-

iohk-bors · 2020-10-01T16:39:57Z

Canceled.

Anviking · 2020-10-01T16:48:13Z

anviking/2175/integration-test-cluster-exposition

Suggestion of direction:

commit 339fe2bf19689613e663da2bbdaec1ea67804a1f (HEAD -> anviking/2175/integration-test-cluster-exposition, origin/anviking/2175/integration-test-cluster-exposition)
Author: Johannes Lund <[email protected]>
Date:   Thu Oct 1 18:40:48 2020 +0200

    Add TestLog to shelley-test-cluster

Anviking

👍

Anviking · 2020-10-05T08:38:45Z

bors r+

@Anviking

2178: expose an executable 'shelley-test-cluster' which starts an integration test cluster with faucets r=Anviking a=KtorZ # Issue Number  #2175 # Overview  - 9722d82 📍 **expose an executable 'shelley-test-cluster' which starts an integration test cluster with faucets** - 6eefd47 📍 **Let the shelley-test-cluster nix build be run from any directory** - 4c4d01e 📍 **Allow setting different LoggerName when configuring trace with 'withLogging'** - 571e67d 📍 **Add TestLog to shelley-test-cluster** - e99d875 📍 **Regenerate nix** # Comments  ~~@Anviking there's probably some clashes with what you're also doing with the logging, so happy to get the logging revision merged first and adjust that one.~~  Co-authored-by: KtorZ <[email protected]> Co-authored-by: Rodney Lorrimar <[email protected]> Co-authored-by: Johannes Lund <[email protected]> Co-authored-by: IOHK <[email protected]>

iohk-bors · 2020-10-05T08:54:21Z

Canceled.

KtorZ · 2020-10-05T11:02:17Z

bors retry

KtorZ · 2020-10-05T11:02:43Z

bors r-

iohk-bors · 2020-10-05T11:02:45Z

Canceled.

…on test cluster with faucets

…ogging'

KtorZ · 2020-10-06T07:38:00Z

bors merge

@Anviking

2178: expose an executable 'shelley-test-cluster' which starts an integration test cluster with faucets r=KtorZ a=KtorZ # Issue Number  #2175 # Overview  - 9722d82 📍 **expose an executable 'shelley-test-cluster' which starts an integration test cluster with faucets** - 6eefd47 📍 **Let the shelley-test-cluster nix build be run from any directory** - 4c4d01e 📍 **Allow setting different LoggerName when configuring trace with 'withLogging'** - 571e67d 📍 **Add TestLog to shelley-test-cluster** - e99d875 📍 **Regenerate nix** # Comments  ~~@Anviking there's probably some clashes with what you're also doing with the logging, so happy to get the logging revision merged first and adjust that one.~~  Co-authored-by: KtorZ <[email protected]> Co-authored-by: Rodney Lorrimar <[email protected]> Co-authored-by: Johannes Lund <[email protected]> Co-authored-by: IOHK <[email protected]>

iohk-bors · 2020-10-06T07:44:17Z

Canceled.

Anviking · 2020-10-06T08:25:59Z

Manually tweaked .nix file

Anviking · 2020-10-06T08:32:29Z

bors r+

iohk-bors · 2020-10-06T09:39:50Z

Build succeeded:

@KtorZ

2235: Fix extra-source-files not containing files used by TH r=hasufell a=hasufell 2239: WIP: Bump version from 2020.9.30 to 2020.10.13 r=KtorZ a=jonathanknowles  Compatible with [`[email protected]`](https://github.com/input-output-hk/jormungandr/releases/tag/v0.9.0) and [`[email protected]`](https://github.com/input-output-hk/cardano-node/releases/tag/1.21.1). ## New Features - Adds the ability for users to select their own SMASH servers for stakepool listings. (#2214) - Adds transaction expiry slots for pending transactions. (#1879) ## Improvements - Adds a 100-wallet scenario to the latency benchmark. (#2223) - Adds an executable `shelley-test-cluster` which starts an integration test cluster that includes faucets. (#2178) - Extends `isOurs` to return the derivation path of an address. (#2219) ## Resolved Issues - Make pool garbage collection handle an unknown current epoch. (#2203) - Fixes incorrect mainnet network parameters returned from API. (#2226) ## Known Issues **_This section is a work in progress._** - Wallet restoration status reported incorrectly on mainnet. ([ADP-483](https://jira.iohk.io/browse/ADP-483)) ## Documentation  Cardano (cardano-node) | ITN (Jörmungandr) --- | --- [API Documentation](https://input-output-hk.github.io/cardano-wallet/api/v2020-10-13) | [API Documentation](https://input-output-hk.github.io/cardano-wallet/api/v2020-10-13) [CLI Manual](https://github.com/input-output-hk/cardano-wallet/wiki/Wallet-command-line-interface/f71ecb7ece3deaec5bf60f4a4beea1d3950e0d46) | [CLI Manual](https://github.com/input-output-hk/cardano-wallet/wiki/Wallet-command-line-interface-jormungandr/f71ecb7ece3deaec5bf60f4a4beea1d3950e0d46) [Docker Manual](https://github.com/input-output-hk/cardano-wallet/wiki/Docker/f71ecb7ece3deaec5bf60f4a4beea1d3950e0d46) | [Docker Manual](https://github.com/input-output-hk/cardano-wallet/wiki/Docker-jormungandr/f71ecb7ece3deaec5bf60f4a4beea1d3950e0d46) ## Installation Instructions ### Cardano (cardano-node) 1. Install [`[email protected]`](https://github.com/input-output-hk/cardano-node/releases/tag/1.21.1). 2. Download the provided `cardano-wallet` for your platform, and uncompress it in a directory that is on your `$PATH`, e.g. `/usr/local/bin`. Or `%PATH%` on Windows. 4. Start `cardano-wallet --help` and see available parameters. #### Docker Pull from DockerHub and verify the version matches 2020.10.13. ``` $ docker pull inputoutput/cardano-wallet:2020.10.13 $ docker run --rm inputoutput/cardano-wallet:2020.10.13 version ``` ### ITN (jormungandr) 1. Install [`[email protected]`](https://github.com/input-output-hk/jormungandr/releases/tag/v0.9.0). 2. Download the provided `cardano-wallet-jormungandr` for your platform, and uncompress it in a directory that is on your `$PATH`, e.g. `/usr/local/bin`. Or `%PATH%` on Windows. 3. (optional) Install the bash/zsh auto-completion script according to the [jormungandr cli manual](https://github.com/input-output-hk/cardano-wallet/wiki/Wallet-Command-Line-Interface/{{JORM_CLI_WIKI_COMMIT}}) 4. Start `cardano-wallet --help` and see available parameters. #### Docker Pull from DockerHub and verify the version matches 2020.10.13 ``` $ docker pull inputoutput/cardano-wallet:2020.10.13-jormungandr $ docker run --rm inputoutput/cardano-wallet:2020.10.13-jormungandr version ``` ## Signatures  Name | Role | Approval --- | --- | ---: Matthias Benkort @KtorZ | Technical Team Lead | ⌛ Piotr Stachyra @piotr-iohk | QA Engineer | ⌛ Tatyana Valkevych @tatyanavych | Release Manager | ⌛ Co-authored-by: Julian Ospald <[email protected]> Co-authored-by: Rodney Lorrimar <[email protected]> Co-authored-by: Jonathan Knowles <[email protected]> Co-authored-by: IOHK <[email protected]>

KtorZ added the ADDING FEATURE Mark a PR as adding a new feature, for auto-generated CHANGELOG label Sep 24, 2020

KtorZ requested a review from Anviking September 24, 2020 09:43

KtorZ self-assigned this Sep 24, 2020

KtorZ force-pushed the KtorZ/2175/integration-test-cluster-exposition branch from 6239764 to 3c3efd7 Compare September 24, 2020 09:56

Anviking reviewed Sep 24, 2020

View reviewed changes

rvl approved these changes Sep 25, 2020

View reviewed changes

Anviking reviewed Sep 29, 2020

View reviewed changes

lib/shelley/exe/shelley-test-cluster.hs Show resolved Hide resolved

KtorZ mentioned this pull request Oct 1, 2020

Explore exposing integration tests cluster for a wider audience #2175

Closed

KtorZ force-pushed the KtorZ/2175/integration-test-cluster-exposition branch from a5f269a to 99d5ecd Compare October 1, 2020 15:16

Anviking reviewed Oct 1, 2020

View reviewed changes

lib/shelley/exe/shelley-test-cluster.hs Outdated Show resolved Hide resolved

KtorZ force-pushed the KtorZ/2175/integration-test-cluster-exposition branch from 99d5ecd to 0d2fcc7 Compare October 1, 2020 15:32

Anviking reviewed Oct 1, 2020

View reviewed changes

lib/shelley/src/Cardano/Wallet/Shelley/Launch.hs Outdated Show resolved Hide resolved

Anviking requested changes Oct 1, 2020

View reviewed changes

KtorZ force-pushed the KtorZ/2175/integration-test-cluster-exposition branch from f3d65b8 to e99d875 Compare October 5, 2020 08:14

Anviking approved these changes Oct 5, 2020

View reviewed changes

Anviking force-pushed the KtorZ/2175/integration-test-cluster-exposition branch from 8da9ccc to 2342b8d Compare October 5, 2020 09:05

expose an executable 'shelley-test-cluster' which starts an integrati…

65da19e

…on test cluster with faucets

rvl and others added 4 commits October 6, 2020 09:37

Let the shelley-test-cluster nix build be run from any directory

9932ee5

Allow setting different LoggerName when configuring trace with 'withL…

f75b43d

…ogging'

Add TestLog to shelley-test-cluster

3a66db5

Regenerate nix

a7b441e

KtorZ force-pushed the KtorZ/2175/integration-test-cluster-exposition branch from 2342b8d to a7b441e Compare October 6, 2020 07:37

fixup! Regenerate nix

937101d

Add support for NO_POOLS in shelley-test-cluster

6a3082d

iohk-bors bot merged commit 4ad068a into master Oct 6, 2020

iohk-bors bot deleted the KtorZ/2175/integration-test-cluster-exposition branch October 6, 2020 09:39

jonathanknowles mentioned this pull request Oct 13, 2020

Bump version from 2020.9.30 to 2020.10.13 #2239

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

expose an executable 'shelley-test-cluster' which starts an integration test cluster with faucets #2178

expose an executable 'shelley-test-cluster' which starts an integration test cluster with faucets #2178

KtorZ commented Sep 24, 2020 •

edited

Loading

Anviking Sep 24, 2020

KtorZ Sep 24, 2020

KtorZ Sep 24, 2020

Anviking Sep 24, 2020

KtorZ Sep 24, 2020

Anviking commented Sep 29, 2020

KtorZ commented Oct 1, 2020

Anviking Oct 1, 2020 •

edited

Loading

KtorZ Oct 1, 2020

Anviking left a comment

Anviking commented Oct 1, 2020

iohk-bors bot commented Oct 1, 2020

Anviking commented Oct 1, 2020

Anviking left a comment

Anviking commented Oct 5, 2020

iohk-bors bot commented Oct 5, 2020

KtorZ commented Oct 5, 2020

KtorZ commented Oct 5, 2020

iohk-bors bot commented Oct 5, 2020

KtorZ commented Oct 6, 2020

iohk-bors bot commented Oct 6, 2020

Anviking commented Oct 6, 2020

Anviking commented Oct 6, 2020

iohk-bors bot commented Oct 6, 2020

expose an executable 'shelley-test-cluster' which starts an integration test cluster with faucets #2178

expose an executable 'shelley-test-cluster' which starts an integration test cluster with faucets #2178

Conversation

KtorZ commented Sep 24, 2020 • edited Loading

Issue Number

Overview

Comments

Anviking Sep 24, 2020

Choose a reason for hiding this comment

KtorZ Sep 24, 2020

Choose a reason for hiding this comment

KtorZ Sep 24, 2020

Choose a reason for hiding this comment

Anviking Sep 24, 2020

Choose a reason for hiding this comment

KtorZ Sep 24, 2020

Choose a reason for hiding this comment

Anviking commented Sep 29, 2020

KtorZ commented Oct 1, 2020

Anviking Oct 1, 2020 • edited Loading

Choose a reason for hiding this comment

KtorZ Oct 1, 2020

Choose a reason for hiding this comment

Anviking left a comment

Choose a reason for hiding this comment

Anviking commented Oct 1, 2020

iohk-bors bot commented Oct 1, 2020

Anviking commented Oct 1, 2020

Anviking left a comment

Choose a reason for hiding this comment

Anviking commented Oct 5, 2020

iohk-bors bot commented Oct 5, 2020

KtorZ commented Oct 5, 2020

KtorZ commented Oct 5, 2020

iohk-bors bot commented Oct 5, 2020

KtorZ commented Oct 6, 2020

iohk-bors bot commented Oct 6, 2020

Anviking commented Oct 6, 2020

Anviking commented Oct 6, 2020

iohk-bors bot commented Oct 6, 2020

KtorZ commented Sep 24, 2020 •

edited

Loading

Anviking Oct 1, 2020 •

edited

Loading