Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve shared wallets #2663

Merged
merged 7 commits into from
May 26, 2021
Merged

Conversation

paweljakubas
Copy link
Contributor

@paweljakubas paweljakubas commented May 21, 2021

Issue Number

adp-934

Overview

  • Adding "incomplete" to status for pending shared wallets
  • comprehensive validation of script templates when posting them
  • integration testing of validation of script templates
  • add GET /shared-wallets/wid/keys and whole infrastructure
  • add GET /wallets/wid/keys
  • unite ApiVerificationKeyShelley and ApiVerificationKeyShared and hence enable with/without hashing capability for shelley style
  • better reuse of code in Api.Link
  • test new endpoints

Comments

Prerequisite : IntersectMBO/cardano-addresses#131

@paweljakubas paweljakubas self-assigned this May 21, 2021
@paweljakubas paweljakubas force-pushed the paweljakubas/adp-934/improve-shared-wallets branch 2 times, most recently from 4be0960 to 4e11b5e Compare May 24, 2021 12:33
@paweljakubas paweljakubas force-pushed the paweljakubas/adp-934/improve-shared-wallets branch 2 times, most recently from c057c29 to 2ed0a9d Compare May 24, 2021 18:49
@paweljakubas paweljakubas requested review from KtorZ, piotr-iohk and rvl May 24, 2021 18:49
@paweljakubas paweljakubas force-pushed the paweljakubas/adp-934/improve-shared-wallets branch from b8e8ffb to 9f93930 Compare May 24, 2021 19:09
@rvl rvl force-pushed the paweljakubas/adp-934/improve-shared-wallets branch from 9291322 to e4a99de Compare May 25, 2021 07:07
Copy link
Contributor

@rvl rvl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 Nice improvements. I have some suggestions though.

, expectField #format (`shouldBe` Extended)
]
let (ApiAccountKeyShared bytes' _) = getFromResponse id aKey
T.decodeUtf8 (hex bytes') `Expectations.shouldBe` accXPubTxt
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The lifted expectations module is meant to replace Test.Hspec - so no need to import it qualified.

Also a util function for hexText would be nice.

@@ -2391,8 +2412,8 @@ data ErrAddCosignerKey
deriving (Eq, Show)

data ErrConstructSharedWallet
= ErrConstructSharedWalletMissingKey
-- ^ The shared wallet' script template doesn't have the wallet's account public key
= ErrConstructSharedWalletWrongScriptTemplate CredentialType Text
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we put this error type into the AddressDiscovery.Shared module with a ToText instance, and Text replaced with ErrValidateScriptTemplate?

type GetAccountKey = "wallets"
:> Capture "walletId" (ApiT WalletId)
:> "keys"
:> QueryParam "extended" Bool
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's probably OK to have hash as a bool query param, but maybe not extended. Didn't we use a sum type elsewhere?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

introduced "format=extended" and "format=non_extended" and used Maybe KeyFormat rather than Maybe Bool

Comment on lines 2350 to 2351
xPubtoBytes :: KeyFormat -> XPub -> ByteString
xPubtoBytes = \case
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That was confusing with the letter case differences. How about:

Suggested change
xPubtoBytes :: KeyFormat -> XPub -> ByteString
xPubtoBytes = \case
publicKeyToBytes' :: WalletKey k => KeyFormat -> k -> ByteString
publickKeyToBytes' f k = case f of
Extended -> xpubToBytes $ getRawKey k
NonExtended -> xpubPublicKey $ getRawKey k

parseRole = \case
hrp | hrp == [humanReadablePart|addr_vk|] -> pure UtxoExternal
hrp | hrp == [humanReadablePart|stake_vk|] -> pure MutableAccount
parseRoleHashing = \case
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How would it look if we get the hrp as text, split on "_" and pattern match on that?

Comment on lines 3114 to 3115
Returned when a user tries to create a shared wallet with script template
that is not validated.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Returned when a user tries to create a shared wallet with script template
that is not validated.
Returned when a user tries to create a shared wallet with script template
that does not pass validation.

code:
type: string
enum: ['shared_wallet_create_not_allowed']
enum: ['shared_wallet_script_template_not_validated']
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not validated = Invalid?

Suggested change
enum: ['shared_wallet_script_template_not_validated']
enum: ['shared_wallet_script_template_invalid']

@@ -4505,7 +4534,7 @@ paths:
summary: Create
description: |
<p align="right">status: <strong>stable</strong></p>
Retrieve account public key from the wallet.
Retrieve any account public key from the wallet provided it was created from mnemonics.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Retrieve any account public key from the wallet provided it was created from mnemonics.
Derive an account public key for any account index. For this key derivation to be possible, the wallet must have been created from mnemonic.

@@ -5441,7 +5489,7 @@ paths:
summary: Create
description: |
<p align="right">status: <strong>stable</strong></p>
Retrieve account public key from the shared wallet.
Derive any account public key from the shared wallet provided the wallet was created from mnemonics.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment as above

summary: Get
description: |
<p align="right">status: <strong>stable</strong></p>
Retrieve account public key of the shared wallet.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Retrieve account public key of the shared wallet.
Retrieve the account public key of this shared wallet.

@paweljakubas paweljakubas force-pushed the paweljakubas/adp-934/improve-shared-wallets branch from e4a99de to cc5f4b0 Compare May 25, 2021 10:04
@piotr-iohk
Copy link
Contributor

@paweljakubas perhaps that's a silly question, but why do we need two separate endpoints for getting account public key? Couldn't this be realized by already existing:

I assume this was done to address:

  1. cannot return account public key for the wallet that's created from acc xpub using https://input-output-hk.github.io/cardano-wallet/api/edge/#operation/postAccountKeyShared
    Following error is returned:
 {"code":"no_root_key","message":"I couldn't find a root private key for the given wallet: 0eb51c11fc2eda7b2b34aa99ceb89bd6044e4015. However, this operation requires that I do have such a key. Either there's no such wallet, or I don't fully own it."}

The same behavior is for shelley wallets. Both have been considered as a bug as per comment -> #2624 (comment)

properties:
id: *walletId
name: *walletName
account_index: *derivationSegment
address_pool_gap: *walletAddressPoolGap
payment_script_template: *scriptTemplate
delegation_script_template: *scriptTemplate
status: *statusIncomplete
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As far as I see the status of complete/active wallet is displayed as:

  "state": {
    "status": "syncing",
   ...
  }

or

  "state": {
    "status": "ready"
  }

Whereas incomplete wallet's status is not wrapped into state and displays only:

"status": "incomplete"

Maybe it would be nice to keep this convention and have:

  "state": {
    "status": "incomplete"
  }

(That's something I'd expect as an API user at least)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@paweljakubas
Copy link
Contributor Author

@piotr-iohk I was thinking long how to fix "posting" account public keys for account based wallets. And get to the conclusion that we should have POST which stands for "deriving" and it means we need mnemonics to do it, so the wallet have to be root key based. Also we can GET the account public key for a given wallet, and it should work for each wallet, account and root based, and also for shared wallets and shelley wallets. I added support for both shelley and shared wallets, also checked in integration testing that it works for root and account based wallets

@paweljakubas
Copy link
Contributor Author

bors r+

@paweljakubas paweljakubas marked this pull request as ready for review May 25, 2021 17:23
@paweljakubas
Copy link
Contributor Author

bors r-

@iohk-bors
Copy link
Contributor

iohk-bors bot commented May 25, 2021

Canceled.

@paweljakubas
Copy link
Contributor Author

bors r+

iohk-bors bot added a commit that referenced this pull request May 25, 2021
2663: Improve shared wallets r=paweljakubas a=paweljakubas

# Issue Number

<!-- Put here a reference to the issue that this PR relates to and which requirements it tackles. Jira issues of the form ADP- will be auto-linked. -->
adp-934

# Overview

<!-- Detail in a few bullet points the work accomplished in this PR -->

- [x] Adding "incomplete" to status for pending shared wallets
- [x] comprehensive validation of script templates when posting them
- [x] integration testing of validation of script templates
- [x] add GET /shared-wallets/wid/keys and whole infrastructure
- [x] add GET /wallets/wid/keys
- [x] unite ApiVerificationKeyShelley and ApiVerificationKeyShared and hence enable with/without hashing capability for shelley style
- [x] better reuse of code in Api.Link
- [x] test new endpoints

# Comments
Prerequisite : IntersectMBO/cardano-addresses#131

<!-- Additional comments or screenshots to attach if any -->

<!--
Don't forget to:

 ✓ Self-review your changes to make sure nothing unexpected slipped through
 ✓ Assign yourself to the PR
 ✓ Assign one or several reviewer(s)
 ✓ Jira will detect and link to this PR once created, but you can also link this PR in the description of the corresponding ticket
 ✓ Acknowledge any changes required to the Wiki
 ✓ Finally, in the PR description delete any empty sections and all text commented in <!--, so that this text does not appear in merge commit messages.
-->


2667: Add property test for `UTxOIndex.selectRandomWithPriority`. r=jonathanknowles a=jonathanknowles

# Issue Number

ADP-890

# Overview

This PR adds a property test for `UTxOIndex.selectRandomWithPriority`.

The `selectRandomWithPriority`  function is designed to:
- select an entry at random from a UTxO index according to a specified list of filter conditions;
- traverse the specified list of filter conditions in order of priority **_from left to right_**.

The test added in this PR provides a basic sanity check to verify that priority order is respected.

# Sample Output

```hs
Cardano.Wallet.Primitive.Types.UTxOIndex
  Indexed UTxO set properties
    Index Selection
      prop_selectRandomWithPriority
        +++ OK, passed 1600 tests:
        59.69% have match for neither asset 1 nor asset 2
        17.12% have match for asset 1 but not for asset 2
        16.31% have match for asset 2 but not for asset 1
         6.88% have match for both asset 1 and asset 2

Finished in 1.0870 seconds
1 example, 0 failures
```

# QA Due Diligence

I ran this test 500 times to increase confidence that it will not fail spuriously. No failures were encountered.


Co-authored-by: Pawel Jakubas <[email protected]>
Co-authored-by: IOHK <[email protected]>
Co-authored-by: Rodney Lorrimar <[email protected]>
Co-authored-by: Jonathan Knowles <[email protected]>
@iohk-bors
Copy link
Contributor

iohk-bors bot commented May 25, 2021

Build failed (retrying...):

#expected

iohk-bors bot added a commit that referenced this pull request May 25, 2021
2663: Improve shared wallets r=paweljakubas a=paweljakubas

# Issue Number

<!-- Put here a reference to the issue that this PR relates to and which requirements it tackles. Jira issues of the form ADP- will be auto-linked. -->
adp-934

# Overview

<!-- Detail in a few bullet points the work accomplished in this PR -->

- [x] Adding "incomplete" to status for pending shared wallets
- [x] comprehensive validation of script templates when posting them
- [x] integration testing of validation of script templates
- [x] add GET /shared-wallets/wid/keys and whole infrastructure
- [x] add GET /wallets/wid/keys
- [x] unite ApiVerificationKeyShelley and ApiVerificationKeyShared and hence enable with/without hashing capability for shelley style
- [x] better reuse of code in Api.Link
- [x] test new endpoints

# Comments
Prerequisite : IntersectMBO/cardano-addresses#131

<!-- Additional comments or screenshots to attach if any -->

<!--
Don't forget to:

 ✓ Self-review your changes to make sure nothing unexpected slipped through
 ✓ Assign yourself to the PR
 ✓ Assign one or several reviewer(s)
 ✓ Jira will detect and link to this PR once created, but you can also link this PR in the description of the corresponding ticket
 ✓ Acknowledge any changes required to the Wiki
 ✓ Finally, in the PR description delete any empty sections and all text commented in <!--, so that this text does not appear in merge commit messages.
-->


Co-authored-by: Pawel Jakubas <[email protected]>
Co-authored-by: IOHK <[email protected]>
Co-authored-by: Rodney Lorrimar <[email protected]>
@iohk-bors
Copy link
Contributor

iohk-bors bot commented May 25, 2021

Build failed:

Failures:

  src/Cardano/Wallet/Api/Link.hs:805:22: 
  1) API Specifications, SHELLEY_ADDRESSES, ANY_ADDRESS_POST_12 - Delegating addresses API roundtrip
       uncaught exception: ErrorCall
       Endpoint not supported for Byron style
       CallStack (from HasCallStack):
         error, called at src/Cardano/Wallet/Api/Link.hs:805:22 in cardano-wallet-core-2021.4.28-CLjD9dqA6KiFsB2TNpTnlI:Cardano.Wallet.Api.Link

  To rerun use: --match "/API Specifications/SHELLEY_ADDRESSES/ANY_ADDRESS_POST_12 - Delegating addresses API roundtrip/"

  src/Cardano/Wallet/Api/Link.hs:805:22: 
  2) API Specifications, SHELLEY_ADDRESSES, POST_ACCOUNT_01 - Can retrieve account public keys
       uncaught exception: ErrorCall
       Endpoint not supported for Byron style
       CallStack (from HasCallStack):
         error, called at src/Cardano/Wallet/Api/Link.hs:805:22 in cardano-wallet-core-2021.4.28-CLjD9dqA6KiFsB2TNpTnlI:Cardano.Wallet.Api.Link

  To rerun use: --match "/API Specifications/SHELLEY_ADDRESSES/POST_ACCOUNT_01 - Can retrieve account public keys/"

  src/Cardano/Wallet/Api/Link.hs:805:22: 
  3) API Specifications, SHELLEY_WALLETS, WALLETS_GET_KEY_01 - golden tests for verification key
       uncaught exception: ErrorCall
       Endpoint not supported for Byron style
       CallStack (from HasCallStack):
         error, called at src/Cardano/Wallet/Api/Link.hs:805:22 in cardano-wallet-core-2021.4.28-CLjD9dqA6KiFsB2TNpTnlI:Cardano.Wallet.Api.Link

  To rerun use: --match "/API Specifications/SHELLEY_WALLETS/WALLETS_GET_KEY_01 - golden tests for verification key/"

  src/Cardano/Wallet/Api/Link.hs:805:22: 
  4) API Specifications, SHELLEY_WALLETS, WALLETS_GET_KEY_02 - invalid index for verification key
       uncaught exception: ErrorCall
       Endpoint not supported for Byron style
       CallStack (from HasCallStack):
         error, called at src/Cardano/Wallet/Api/Link.hs:805:22 in cardano-wallet-core-2021.4.28-CLjD9dqA6KiFsB2TNpTnlI:Cardano.Wallet.Api.Link

  To rerun use: --match "/API Specifications/SHELLEY_WALLETS/WALLETS_GET_KEY_02 - invalid index for verification key/"

  src/Cardano/Wallet/Api/Link.hs:805:22: 
  5) API Specifications, SHELLEY_WALLETS, WALLETS_GET_KEY_03 - unknown wallet
       uncaught exception: ErrorCall
       Endpoint not supported for Byron style
       CallStack (from HasCallStack):
         error, called at src/Cardano/Wallet/Api/Link.hs:805:22 in cardano-wallet-core-2021.4.28-CLjD9dqA6KiFsB2TNpTnlI:Cardano.Wallet.Api.Link

  To rerun use: --match "/API Specifications/SHELLEY_WALLETS/WALLETS_GET_KEY_03 - unknown wallet/"

  src/Cardano/Wallet/Api/Link.hs:805:22: 
  6) API Specifications, SHELLEY_WALLETS, WALLETS_SIGNATURES_01 - can verify signature
       uncaught exception: ErrorCall
       Endpoint not supported for Byron style
       CallStack (from HasCallStack):
         error, called at src/Cardano/Wallet/Api/Link.hs:805:22 in cardano-wallet-core-2021.4.28-CLjD9dqA6KiFsB2TNpTnlI:Cardano.Wallet.Api.Link

  To rerun use: --match "/API Specifications/SHELLEY_WALLETS/WALLETS_SIGNATURES_01 - can verify signature/"

Randomized with seed 576096783

Finished in 1554.2501 seconds
796 examples, 6 failures, 15 pending
builder for '/nix/store/9yz0fwd9is5gmq0mhanpld5vqn60d3ry-cardano-wallet-test-integration-2021.4.28-x86_64-unknown-linux-musl-check-x86_64-unknown-linux-musl.drv' failed with exit code 1

These failures look legit.
#expected

@rvl
Copy link
Contributor

rvl commented May 26, 2021

@piotr-iohk Thanks for the question about multiple API endpoints which do similar things. This is not a silly question at all and it's good to take a step back and assess the API as a whole, like you have done.

To add to @paweljakubas answer, yes they do slightly different things. The POST one can derive an account public key for any index. To do this, it must be supplied with the password to decrypt the root key. The GET one can only return the "default" or "current" account public key for this wallet. That key is already cached in the wallet state, so no need to derive anything, and therefore no need for a password.

Then we get on to the mechanics of HTTP. It is generally discouraged to put passwords in GET query parameters. The preferred way is to use a HTTP header or put the password into the request body of a POST request (the latter often being easier for API users). However, these key derivation endpoints are really GETs, semantically speaking.

Anyway, your comment convinced me that it is less confusing to have only one account key endpoint, with optional parameters. It has to be POST. After this PR, I would like to spec out a streamlined API for deriving, which will save us work in future.

@rvl rvl force-pushed the paweljakubas/adp-934/improve-shared-wallets branch from 7bda368 to a74d89c Compare May 26, 2021 05:14
@rvl
Copy link
Contributor

rvl commented May 26, 2021

I squashed the git history a bit, rebased, made a minor stylistic change to swagger.yaml, and fixed the failing integration tests.

bors r+

@piotr-iohk
Copy link
Contributor

@rvl @paweljakubas thanks for your responses.

Anyway, your comment convinced me that it is less confusing to have only one account key endpoint, with optional parameters. It has to be POST. After this PR, I would like to spec out a streamlined API for deriving, which will save us work in future.

@rvl is this something we'd like to tackle as part of ADP-934 or later?

@rvl
Copy link
Contributor

rvl commented May 26, 2021

I think it can be a separate ticket, so we don't hold up the address derivation part of multisig.

@iohk-bors
Copy link
Contributor

iohk-bors bot commented May 26, 2021

Build succeeded:

@iohk-bors iohk-bors bot merged commit 1e89c9a into master May 26, 2021
@iohk-bors iohk-bors bot deleted the paweljakubas/adp-934/improve-shared-wallets branch May 26, 2021 06:28
WilliamKingNoel-Bot pushed a commit that referenced this pull request May 26, 2021
2663: Improve shared wallets r=rvl a=paweljakubas

# Issue Number

<!-- Put here a reference to the issue that this PR relates to and which requirements it tackles. Jira issues of the form ADP- will be auto-linked. -->
adp-934

# Overview

<!-- Detail in a few bullet points the work accomplished in this PR -->

- [x] Adding "incomplete" to status for pending shared wallets
- [x] comprehensive validation of script templates when posting them
- [x] integration testing of validation of script templates
- [x] add GET /shared-wallets/wid/keys and whole infrastructure
- [x] add GET /wallets/wid/keys
- [x] unite ApiVerificationKeyShelley and ApiVerificationKeyShared and hence enable with/without hashing capability for shelley style
- [x] better reuse of code in Api.Link
- [x] test new endpoints

# Comments
Prerequisite : IntersectMBO/cardano-addresses#131

<!-- Additional comments or screenshots to attach if any -->

<!--
Don't forget to:

 ✓ Self-review your changes to make sure nothing unexpected slipped through
 ✓ Assign yourself to the PR
 ✓ Assign one or several reviewer(s)
 ✓ Jira will detect and link to this PR once created, but you can also link this PR in the description of the corresponding ticket
 ✓ Acknowledge any changes required to the Wiki
 ✓ Finally, in the PR description delete any empty sections and all text commented in <!--, so that this text does not appear in merge commit messages.
-->

Co-authored-by: Pawel Jakubas <[email protected]>
Co-authored-by: Rodney Lorrimar <[email protected]>
Co-authored-by: IOHK <[email protected]> 1e89c9a
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants