Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

API data-types & specifications for arbitrary metadata signing. #2257

Merged
merged 6 commits into from
Oct 21, 2020

Conversation

KtorZ
Copy link
Member

@KtorZ KtorZ commented Oct 19, 2020

Issue Number

ADP-481 > ADP-508

Overview

  • 209781b
    📍 define ToText/FromText instances for 'DerivationIndex'
    I had to also move it in '...Primitive.AddressDerivation' because of otherwise circular dependencies (since this module is using the Bounded instances of 'Index', relying on the 'DerivationType' as type parameter. This is mostly extracting the text parsing that was already done in the JSON instance, and moving it as a ToText/FromText instances (and re-using it to build the JSON instances). I've however added a quick sanity text roundtrip property test in the meantime.

  • c965936
    📍 rename UTxOternal -> Utxoternal
    We use the data-type constructor in order to generate text instances.
    However, our automatic casing function is doing pretty bad with the
    mixed cases and we end up with something like:

    • u_tx_o_external
    • u_tx_o_internal
    • mutable_account

    Instead of fixing the instance itself to work around this, I found it
    easier to just fix the type (which isn't used in many places).

  • c4cbafe
    📍 do not output back invalid input when parsing text values.
    This is a rather bad practice; if a value failed to parse, then we have very little idea about what it is and what's it's shape. It may not be even readable / printable. Instead, the error message should show valid expected options.

  • fb5ab0d
    📍 extend API and specifications to include the 'signMetadata' operation

Comments

@KtorZ KtorZ added the ADDING FEATURE Mark a PR as adding a new feature, for auto-generated CHANGELOG label Oct 19, 2020
@KtorZ KtorZ requested a review from rvl October 19, 2020 15:19
@KtorZ KtorZ self-assigned this Oct 19, 2020
Just s ->
pure s
parseJSON = parseJSON
>=> fmap ApiT . eitherToParser . first ShowFmt . fromText
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Parsing delegated to the FromText instance and moved near the data-type declaration.

where
firstHardened = getIndex @'Hardened minBound

toJSON = toJSON . toText . getApiT
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Serialization delegated to the ToText instance and moved near the data-type declaration.

@KtorZ KtorZ force-pushed the KtorZ/ADP-508/metadata-api-data-types branch from da28646 to e03e3fe Compare October 19, 2020 15:22

> ⚠️ On the incentivized testnet, this parameter is not requred, but rather completely ignored.
> ⚠️ On the incentivized testnet, this parameter is not requred, but rather completely ignored.
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was somewhat indented too far. Sorry for the noise.

Copy link
Contributor

@rvl rvl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Comment on lines +172 to +173
= UtxoExternal
| UtxoInternal
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this could be fixed in fromTextToBoundedEnum SnakeLowerCase.
It's spelt UTxO everywhere else.
This important word deserves special case treatment, so that we don't need to mangle our type names just to get the preferred serialisation.

Copy link
Member Author

@KtorZ KtorZ Oct 20, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought of both options and thought that it was cleaner in the end to rename the type here. It's perhaps a little less pervasive as a change to just deal with it in the toText//fromText instances. Any way requires a database fix / migration / work-around :|


paths:
/wallets/{walletId}/{role}/{index}/signatures:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

KtorZ added 4 commits October 20, 2020 14:04
  I had to also move it in '...Primitive.AddressDerivation' because of otherwise circular dependencies (since this module is using the Bounded instances of 'Index', relying on the 'DerivationType' as type parameter. This is mostly extracting the text parsing that was already done in the JSON instance, and moving it as a ToText/FromText instances (and re-using it to build the JSON instances). I've however added a quick sanity text roundtrip property test in the meantime.
  We use the data-type constructor in order to generate text instances.
  However, our automatic casing function is doing pretty bad with the
  mixed cases and we end up with something like:

    - u_tx_o_external
    - u_tx_o_internal
    - mutable_account

  Instead of fixing the instance itself to work around this, I found it
  easier to just fix the type (which isn't used in many places).
  This is a rather bad practice; if a value failed to parse, then we have very little idea about _what_ it is and what's it's shape. It may not be even readable / printable. Instead, the error message should show valid expected options.
@KtorZ KtorZ force-pushed the KtorZ/ADP-508/metadata-api-data-types branch from e03e3fe to fb5ab0d Compare October 20, 2020 16:06
@KtorZ
Copy link
Member Author

KtorZ commented Oct 20, 2020

bors merge

iohk-bors bot added a commit that referenced this pull request Oct 20, 2020
2257: API data-types & specifications for arbitrary metadata signing. r=KtorZ a=KtorZ

# Issue Number

<!-- Put here a reference to the issue this PR relates to and which requirements it tackles -->

ADP-481 > ADP-508

# Overview

<!-- Detail in a few bullet points the work accomplished in this PR -->

- 209781b
  📍 **define ToText/FromText instances for 'DerivationIndex'**
    I had to also move it in '...Primitive.AddressDerivation' because of otherwise circular dependencies (since this module is using the Bounded instances of 'Index', relying on the 'DerivationType' as type parameter. This is mostly extracting the text parsing that was already done in the JSON instance, and moving it as a ToText/FromText instances (and re-using it to build the JSON instances). I've however added a quick sanity text roundtrip property test in the meantime.

- c965936
  📍 **rename UTxO*ternal -> Utxo*ternal**
    We use the data-type constructor in order to generate text instances.
  However, our automatic casing function is doing pretty bad with the
  mixed cases and we end up with something like:

    - u_tx_o_external
    - u_tx_o_internal
    - mutable_account

  Instead of fixing the instance itself to work around this, I found it
  easier to just fix the type (which isn't used in many places).

- c4cbafe
  📍 **do not output back invalid input when parsing text values.**
    This is a rather bad practice; if a value failed to parse, then we have very little idea about _what_ it is and what's it's shape. It may not be even readable / printable. Instead, the error message should show valid expected options.

- fb5ab0d
  📍 **extend API and specifications to include the 'signMetadata' operation**

# Comments

<!-- Additional comments or screenshots to attach if any -->

<!-- 
Don't forget to:

 ✓ Self-review your changes to make sure nothing unexpected slipped through
 ✓ Assign yourself to the PR
 ✓ Assign one or several reviewer(s)
 ✓ Once created, link this PR to its corresponding ticket
 ✓ Assign the PR to a corresponding milestone
 ✓ Acknowledge any changes required to the Wiki
-->


Co-authored-by: KtorZ <[email protected]>
@iohk-bors
Copy link
Contributor

iohk-bors bot commented Oct 20, 2020

Build failed:

1) API Specifications, SHELLEY_ADDRESSES, ADDRESS_LIST_02 - Invalid filters are bad requests, usedd
--
  | "Error in $: parsing [] failed, expected Array, but encountered Object: Response {responseStatus = Status {statusCode = 400, statusMessage = \"Bad Request\"}, responseVersion = HTTP/1.1, responseHeaders = [(\"Transfer-Encoding\",\"chunked\"),(\"Date\",\"Tue, 20 Oct 2020 16:33:46 GMT\"),(\"Server\",\"Warp/3.3.5\"),(\"Content-Type\",\"application/json;charset=utf-8\")], responseBody = \"{\\\"code\\\":\\\"bad_request\\\",\\\"message\\\":\\\"Error parsing query parameter state failed: Unable to decode the given text value. Please specify one of the following values: used, unused.\\\"}\", responseCookieJar = CJ {expose = []}, responseClose' = ResponseClose}" does not contain "Error parsing query parameter state failed: Unable to decode the given value: 'usedd'. Please specify one of the following values: used, unused."
  | While verifying (Status {statusCode = 400, statusMessage = "Bad Request"},Left (DecodeFailure "Error in $: parsing [] failed, expected Array, but encountered Object: Response {responseStatus = Status {statusCode = 400, statusMessage = \"Bad Request\"}, responseVersion = HTTP/1.1, responseHeaders = [(\"Transfer-Encoding\",\"chunked\"),(\"Date\",\"Tue, 20 Oct 2020 16:33:46 GMT\"),(\"Server\",\"Warp/3.3.5\"),(\"Content-Type\",\"application/json;charset=utf-8\")], responseBody = \"{\\\"code\\\":\\\"bad_request\\\",\\\"message\\\":\\\"Error parsing query parameter state failed: Unable to decode the given text value. Please specify one of the following values: used, unused.\\\"}\", responseCookieJar = CJ {expose = []}, responseClose' = ResponseClose}"))

plus a few more similar looking failures

@rvl
Copy link
Contributor

rvl commented Oct 21, 2020

Fixed the integration test - baffling failure message though.

bors r+

iohk-bors bot added a commit that referenced this pull request Oct 21, 2020
2257: API data-types & specifications for arbitrary metadata signing. r=rvl a=KtorZ

# Issue Number

<!-- Put here a reference to the issue this PR relates to and which requirements it tackles -->

ADP-481 > ADP-508

# Overview

<!-- Detail in a few bullet points the work accomplished in this PR -->

- 209781b
  📍 **define ToText/FromText instances for 'DerivationIndex'**
    I had to also move it in '...Primitive.AddressDerivation' because of otherwise circular dependencies (since this module is using the Bounded instances of 'Index', relying on the 'DerivationType' as type parameter. This is mostly extracting the text parsing that was already done in the JSON instance, and moving it as a ToText/FromText instances (and re-using it to build the JSON instances). I've however added a quick sanity text roundtrip property test in the meantime.

- c965936
  📍 **rename UTxO*ternal -> Utxo*ternal**
    We use the data-type constructor in order to generate text instances.
  However, our automatic casing function is doing pretty bad with the
  mixed cases and we end up with something like:

    - u_tx_o_external
    - u_tx_o_internal
    - mutable_account

  Instead of fixing the instance itself to work around this, I found it
  easier to just fix the type (which isn't used in many places).

- c4cbafe
  📍 **do not output back invalid input when parsing text values.**
    This is a rather bad practice; if a value failed to parse, then we have very little idea about _what_ it is and what's it's shape. It may not be even readable / printable. Instead, the error message should show valid expected options.

- fb5ab0d
  📍 **extend API and specifications to include the 'signMetadata' operation**

# Comments

<!-- Additional comments or screenshots to attach if any -->

<!-- 
Don't forget to:

 ✓ Self-review your changes to make sure nothing unexpected slipped through
 ✓ Assign yourself to the PR
 ✓ Assign one or several reviewer(s)
 ✓ Once created, link this PR to its corresponding ticket
 ✓ Assign the PR to a corresponding milestone
 ✓ Acknowledge any changes required to the Wiki
-->


Co-authored-by: KtorZ <[email protected]>
Co-authored-by: Rodney Lorrimar <[email protected]>
@iohk-bors
Copy link
Contributor

iohk-bors bot commented Oct 21, 2020

Build failed:

hlint,
#expected

@rvl
Copy link
Contributor

rvl commented Oct 21, 2020

bors r+

iohk-bors bot added a commit that referenced this pull request Oct 21, 2020
2244: Add derivation paths to coin selection change outputs r=jonathanknowles a=jonathanknowles

# Issue Number

#2247 
  
# Overview

This PR introduces a new `ApiCoinSelectionChange` type that includes a derivation path:

```hs
data ApiCoinSelectionChange (n :: NetworkDiscriminant) = ApiCoinSelectionChange
    { address :: !(ApiT Address, Proxy n)
    , amount :: !(Quantity "lovelace" Natural)
    , derivationPath :: NonEmpty (ApiT DerivationIndex)
    } deriving (Eq, Generic, Show)
```

In addition, we modify the `ApiCoinSelection` type so that ordinary outputs (**_without_** derivation paths) and change outputs (**_with_** derivation paths) are held in separate fields:

```patch

data ApiCoinSelection (n :: NetworkDiscriminant) = ApiCoinSelection
    { inputs :: !(NonEmpty (ApiCoinSelectionInput n))
-   , outputs :: !(NonEmpty (AddressAmount (ApiT Address, Proxy n)))
+   , outputs :: !(NonEmpty (ApiCoinSelectionOutput n))
+   , change :: ![ApiCoinSelectionChange n]
    } deriving (Eq, Generic, Show)

```
# Comments

This PR updates the following integration tests:

- `WALLETS_COIN_SELECTION_01`.
- `WALLETS_COIN_SELECTION_02`.
- `SHELLEY_HW_WALLETS/HW_WALLETS_04`.
- `BYRON_HW_WALLETS/HW_WALLETS_04`.

2257: API data-types & specifications for arbitrary metadata signing. r=rvl a=KtorZ

# Issue Number

<!-- Put here a reference to the issue this PR relates to and which requirements it tackles -->

ADP-481 > ADP-508

# Overview

<!-- Detail in a few bullet points the work accomplished in this PR -->

- 209781b
  📍 **define ToText/FromText instances for 'DerivationIndex'**
    I had to also move it in '...Primitive.AddressDerivation' because of otherwise circular dependencies (since this module is using the Bounded instances of 'Index', relying on the 'DerivationType' as type parameter. This is mostly extracting the text parsing that was already done in the JSON instance, and moving it as a ToText/FromText instances (and re-using it to build the JSON instances). I've however added a quick sanity text roundtrip property test in the meantime.

- c965936
  📍 **rename UTxO*ternal -> Utxo*ternal**
    We use the data-type constructor in order to generate text instances.
  However, our automatic casing function is doing pretty bad with the
  mixed cases and we end up with something like:

    - u_tx_o_external
    - u_tx_o_internal
    - mutable_account

  Instead of fixing the instance itself to work around this, I found it
  easier to just fix the type (which isn't used in many places).

- c4cbafe
  📍 **do not output back invalid input when parsing text values.**
    This is a rather bad practice; if a value failed to parse, then we have very little idea about _what_ it is and what's it's shape. It may not be even readable / printable. Instead, the error message should show valid expected options.

- fb5ab0d
  📍 **extend API and specifications to include the 'signMetadata' operation**

# Comments

<!-- Additional comments or screenshots to attach if any -->

<!-- 
Don't forget to:

 ✓ Self-review your changes to make sure nothing unexpected slipped through
 ✓ Assign yourself to the PR
 ✓ Assign one or several reviewer(s)
 ✓ Once created, link this PR to its corresponding ticket
 ✓ Assign the PR to a corresponding milestone
 ✓ Acknowledge any changes required to the Wiki
-->


Co-authored-by: Jonathan Knowles <[email protected]>
Co-authored-by: KtorZ <[email protected]>
Co-authored-by: Rodney Lorrimar <[email protected]>
@iohk-bors
Copy link
Contributor

iohk-bors bot commented Oct 21, 2020

This PR was included in a batch that timed out, it will be automatically retried

iohk-bors bot added a commit that referenced this pull request Oct 21, 2020
2257: API data-types & specifications for arbitrary metadata signing. r=rvl a=KtorZ

# Issue Number

<!-- Put here a reference to the issue this PR relates to and which requirements it tackles -->

ADP-481 > ADP-508

# Overview

<!-- Detail in a few bullet points the work accomplished in this PR -->

- 209781b
  📍 **define ToText/FromText instances for 'DerivationIndex'**
    I had to also move it in '...Primitive.AddressDerivation' because of otherwise circular dependencies (since this module is using the Bounded instances of 'Index', relying on the 'DerivationType' as type parameter. This is mostly extracting the text parsing that was already done in the JSON instance, and moving it as a ToText/FromText instances (and re-using it to build the JSON instances). I've however added a quick sanity text roundtrip property test in the meantime.

- c965936
  📍 **rename UTxO*ternal -> Utxo*ternal**
    We use the data-type constructor in order to generate text instances.
  However, our automatic casing function is doing pretty bad with the
  mixed cases and we end up with something like:

    - u_tx_o_external
    - u_tx_o_internal
    - mutable_account

  Instead of fixing the instance itself to work around this, I found it
  easier to just fix the type (which isn't used in many places).

- c4cbafe
  📍 **do not output back invalid input when parsing text values.**
    This is a rather bad practice; if a value failed to parse, then we have very little idea about _what_ it is and what's it's shape. It may not be even readable / printable. Instead, the error message should show valid expected options.

- fb5ab0d
  📍 **extend API and specifications to include the 'signMetadata' operation**

# Comments

<!-- Additional comments or screenshots to attach if any -->

<!-- 
Don't forget to:

 ✓ Self-review your changes to make sure nothing unexpected slipped through
 ✓ Assign yourself to the PR
 ✓ Assign one or several reviewer(s)
 ✓ Once created, link this PR to its corresponding ticket
 ✓ Assign the PR to a corresponding milestone
 ✓ Acknowledge any changes required to the Wiki
-->


Co-authored-by: KtorZ <[email protected]>
Co-authored-by: Rodney Lorrimar <[email protected]>
@iohk-bors
Copy link
Contributor

iohk-bors bot commented Oct 21, 2020

Build failed:


Failures:

  src/Test/Integration/Scenario/CLI/Shelley/Addresses.hs:181:13: 
  1) CLI Specifications, SHELLEY_CLI_ADDRESSES, ADDRESS_LIST_03 - Generates new address pool gap
       expected: ExitSuccess
        but got: ExitFailure 1

  To rerun use: --match "/CLI Specifications/SHELLEY_CLI_ADDRESSES/ADDRESS_LIST_03 - Generates new address pool gap/"

@KtorZ
Copy link
Member Author

KtorZ commented Oct 21, 2020

bors retry

@iohk-bors
Copy link
Contributor

iohk-bors bot commented Oct 21, 2020

Build succeeded:

@iohk-bors iohk-bors bot merged commit 61b6dc9 into master Oct 21, 2020
@iohk-bors iohk-bors bot deleted the KtorZ/ADP-508/metadata-api-data-types branch October 21, 2020 15:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ADDING FEATURE Mark a PR as adding a new feature, for auto-generated CHANGELOG
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants