Fetch metadata discovered on-chain from remote-peers #1776

KtorZ · 2020-06-17T17:20:22Z

Issue Number

Overview

854859c
📍 add logic for fetching metadata from references in the database
The workflow is as follows:
Metadata are discovered on chain in pool registration certificates by
the stake pool monitoring loop. Every time a new metadata URL and hash
are found in a certificate, they are stored in a specific table which
acts as a queue.
Then, another worker loop watches consumes items from that queue and
fetch metadata corresponding to each item and stores the result in
another table for long-term caching.

Note: Metadata can't be updated freely since the hash of their content
is stored on-chain as part of the registration certificate. New
metadata will necessarily means a new certificate will show up, and
will go over the whole fetching loop again.

I've currently abstracted away the part that is actually fetching the
metadata so that, in a first time, we can directly fetch from remote
servers directly, whereas in a second time, we can replace this with a
deployed centralized aggregation server like SMASH.

3ed5c94
📍 add and serve metadata in integration cluster pool certificates
25d3895
📍 wrap raw 'Text' as 'StakePoolMetadataUrl' newtype
In order to define some necessary non-orphan instances for persitent.
385c015
📍 add method for storing metadata and getting next one to fetch from database
I'll add some property-based testing to make sure this works fine,
in particular this is using raw sql to avoid having to do too many queries
with too many things stored in-memory.

Comments

In progress, mostly untested yet.

[cardano-wallet.pools-engine:Info:192] [2020-06-18 13:12:14.36 UTC] Taking a little break from fetching metadata, back to it in about 0.2s
[cardano-wallet.pools-engine:Info:190] [2020-06-18 13:12:14.36 UTC] Applying blocks [0.0 ... 0.0]
[cardano-wallet.pools-engine:Info:190] [2020-06-18 13:12:14.36 UTC] Applying blocks [0.12 ... 2.0]
[cardano-wallet.pools-engine:Info:190] [2020-06-18 13:12:14.36 UTC] Discovered stake pool registration: Registration of c7258ccc owned by [ed25519_pk1va7rf7f7p097545q8nplgwz98n7x70zk0qm8vhfx3q2apedlt7fsrttw4c]
[cardano-wallet.pools-engine:Info:190] [2020-06-18 13:12:14.37 UTC] Discovered stake pool registration: Registration of 775af3b2 owned by [ed25519_pk1nl63hryqlfljan9046ztd6ejgfersnzevk6cvy2rlkr2npkmdvcqvezqk6]
[cardano-wallet.pools-engine:Info:190] [2020-06-18 13:12:14.37 UTC] Discovered stake pool registration: Registration of 5a7b67c7 owned by [ed25519_pk1vyvapw37ywgugefz0cl9580272w5399m4u4d74mgrwu7st5dd20qsgg22z]
[cardano-wallet.pools-engine:Info:192] [2020-06-18 13:12:14.56 UTC] Fetching metadata for pool c7258ccc from http://localhost:34159/metadata.json
[cardano-wallet.pools-engine:Info:192] [2020-06-18 13:12:14.56 UTC] Successfully fetched metadata for pool c7258ccc: StakePoolMetadata {ticker = StakePoolTicker {unStakePoolTicker = "GPA"}, name = "Genesis Pool A", description = Nothing, homepage = "https://iohk.io"}
[cardano-wallet.pools-engine:Info:192] [2020-06-18 13:12:14.57 UTC] Fetching metadata for pool 775af3b2 from http://localhost:40195/metadata.json
[cardano-wallet.pools-engine:Info:192] [2020-06-18 13:12:14.57 UTC] Successfully fetched metadata for pool 775af3b2: StakePoolMetadata {ticker = StakePoolTicker {unStakePoolTicker = "GPB"}, name = "Genesis Pool B", description = Nothing, homepage = "https://iohk.io"}
[cardano-wallet.pools-engine:Info:192] [2020-06-18 13:12:14.57 UTC] Fetching metadata for pool 5a7b67c7 from http://localhost:45173/metadata.json
[cardano-wallet.pools-engine:Info:192] [2020-06-18 13:12:14.57 UTC] Successfully fetched metadata for pool 5a7b67c7: StakePoolMetadata {ticker = StakePoolTicker {unStakePoolTicker = "GPC"}, name = "Genesis Pool C", description = Just "Lorem Ipsum Dolor Sit Amet.", homepage = "https://iohk.io"}
[cardano-wallet.pools-engine:Info:192] [2020-06-18 13:12:14.57 UTC] Taking a little break from fetching metadata, back to it in about 0.2s

lib/core/src/Cardano/Pool/DB.hs

Anviking · 2020-06-18T15:07:23Z

lib/shelley/src/Cardano/Wallet/Shelley/Pools.hs

+        traceWith tr $ MsgFetchPoolMetadata pid url
+        fetchMetadata url hash >>= \case
+            Left msg -> Nothing <$ do
+                traceWith tr $ MsgFetchPoolMetadataFailure pid msg


Wouldn't we want to store the failure in the db to avoid the worker retrying bad links forever?

Or are you handling it? 🤔

Wouldn't we want to store the failure in the db to avoid the worker retrying bad links forever?

For now, failures are just retried every next block. I'll do something a little smarter in another PR. My plan is to keep in-memory some indexes with the failing ones to avoid retrying them more than X every hour. I don't think it's necessary to store anything in the database here, if the wallet is restarted, it'll retry right away and then wait again until the next hour before re-retrying. I see it as a nice feature :)

The workflow is as follows: - Metadata are discovered on chain in pool registration certificates by the stake pool monitoring loop. Every time a new metadata URL and hash are found in a certificate, they are stored in a specific table which acts as a queue. - Then, another worker loop watches consumes items from that queue and fetch metadata corresponding to each item and stores the result in another table for long-term caching. Note: Metadata can't be updated freely since the hash of their content is stored on-chain as part of the registration certificate. New metadata will necessarily means a new certificate will show up, and will go over the whole fetching loop again. I've currently abstracted away the part that is actually fetching the metadata so that, in a first time, we can directly fetch from remote servers directly, whereas in a second time, we can replace this with a deployed centralized aggregation server like SMASH.

In order to define some necessary non-orphan instances for persitent.

…tabase I'll add some property-based testing to make sure this works fine, in particular this is using raw sql to avoid having to do too many queries with too many things stored in-memory.

Anviking · 2020-06-18T17:21:00Z

lib/core/src/Cardano/Wallet/Primitive/Types.hs

@@ -613,6 +627,21 @@ data StakePoolMetadata = StakePoolMetadata
    -- ^ Absolute URL for the stake pool's homepage link.
    } deriving (Eq, Show, Generic)

+instance FromJSON StakePoolMetadata where
+    parseJSON = withObject "StakePoolMetadta" $ \obj -> do


Suggested change

parseJSON = withObject "StakePoolMetadta" $ \obj -> do

parseJSON = withObject "StakePoolMetadata" $ \obj -> do

Anviking · 2020-06-18T17:26:40Z

lib/core/test/unit/Cardano/Pool/DB/Properties.hs

+            [ "Read from DB (" <> show (length refs) <> "): " <> show refs
+            ]
+        assertWith "no more than known entries"
+            (length refs <= length entries)


(Nitpick) Superfluous assertion given the next check? Or is this on purpose to fail fast?

Hmm, indeed, I don't think both assertions below can be true and this one false. so it looks like redundant.

Although, perhaps some redundancy is good for friendlier failure reasons 🤷‍♂️

Anviking · 2020-06-18T17:36:29Z

lib/core/test/unit/Cardano/Pool/DB/Properties.hs

+                [ "Read from DB (" <> show (length refs') <> "): " <> show refs'
+                ]
+            assertWith "fetched metadata isn't listed anymore"
+                (hash `notElem` ((\(_,_,c) -> c) <$> refs'))


This seems like an important property, but the names prop_unfetchedPoolMetadataRefs and propB makes it difficult to see.

Maybe just merging propA and propB would make it clearer.

Maybe just merging propA and propB would make it clearer.

There's nothing really to merge. In theory, we should really have two distinct properties but I was a bit lazy to rewrite the setup and function signature so I just added it quickly after. I could name propA and propB a bit better.

lib/core/test/unit/Cardano/Pool/DB/Properties.hs

Anviking · 2020-06-18T17:46:56Z

lib/shelley/src/Cardano/Wallet/Shelley/Pools.hs

+    -- If there's no metadata, we typically need not to retry sooner than the
+    -- next block. So waiting for a delay that is roughly the same order of
+    -- magnitude as the (slot length / active slot coeff) sounds sound.
+    blockFrequency = ceiling (1/f) * toMicroSecond slotLength


Don't you want a $ after ceiling?

λ> f 9.0e-2 λ> sl 10 λ> (1/f) * sl 111.11111111111111 λ> ceiling (1/f) * sl 120

This is an order of magnitude, so it really doesn't matter.

…tions everywhere

…om genesis

…model

KtorZ · 2020-06-18T18:13:39Z

bors r+

1776: Fetch metadata discovered on-chain from remote-peers r=KtorZ a=KtorZ # Issue Number  # Overview  - 854859c 📍 **add logic for fetching metadata from references in the database** The workflow is as follows: - Metadata are discovered on chain in pool registration certificates by the stake pool monitoring loop. Every time a new metadata URL and hash are found in a certificate, they are stored in a specific table which acts as a queue. - Then, another worker loop watches consumes items from that queue and fetch metadata corresponding to each item and stores the result in another table for long-term caching. Note: Metadata can't be updated freely since the hash of their content is stored on-chain as part of the registration certificate. New metadata will necessarily means a new certificate will show up, and will go over the whole fetching loop again. I've currently abstracted away the part that is actually fetching the metadata so that, in a first time, we can directly fetch from remote servers directly, whereas in a second time, we can replace this with a deployed centralized aggregation server like SMASH. - 3ed5c94 📍 **add and serve metadata in integration cluster pool certificates** - 25d3895 📍 **wrap raw 'Text' as 'StakePoolMetadataUrl' newtype** In order to define some necessary non-orphan instances for persitent. - 385c015 📍 **add method for storing metadata and getting next one to fetch from database** I'll add some property-based testing to make sure this works fine, in particular this is using raw sql to avoid having to do too many queries with too many things stored in-memory. # Comments  In progress, mostly untested yet. ``` [cardano-wallet.pools-engine:Info:192] [2020-06-18 13:12:14.36 UTC] Taking a little break from fetching metadata, back to it in about 0.2s [cardano-wallet.pools-engine:Info:190] [2020-06-18 13:12:14.36 UTC] Applying blocks [0.0 ... 0.0] [cardano-wallet.pools-engine:Info:190] [2020-06-18 13:12:14.36 UTC] Applying blocks [0.12 ... 2.0] [cardano-wallet.pools-engine:Info:190] [2020-06-18 13:12:14.36 UTC] Discovered stake pool registration: Registration of c7258ccc owned by [ed25519_pk1va7rf7f7p097545q8nplgwz98n7x70zk0qm8vhfx3q2apedlt7fsrttw4c] [cardano-wallet.pools-engine:Info:190] [2020-06-18 13:12:14.37 UTC] Discovered stake pool registration: Registration of 775af3b2 owned by [ed25519_pk1nl63hryqlfljan9046ztd6ejgfersnzevk6cvy2rlkr2npkmdvcqvezqk6] [cardano-wallet.pools-engine:Info:190] [2020-06-18 13:12:14.37 UTC] Discovered stake pool registration: Registration of 5a7b67c7 owned by [ed25519_pk1vyvapw37ywgugefz0cl9580272w5399m4u4d74mgrwu7st5dd20qsgg22z] [cardano-wallet.pools-engine:Info:192] [2020-06-18 13:12:14.56 UTC] Fetching metadata for pool c7258ccc from http://localhost:34159/metadata.json [cardano-wallet.pools-engine:Info:192] [2020-06-18 13:12:14.56 UTC] Successfully fetched metadata for pool c7258ccc: StakePoolMetadata {ticker = StakePoolTicker {unStakePoolTicker = "GPA"}, name = "Genesis Pool A", description = Nothing, homepage = "https://iohk.io"} [cardano-wallet.pools-engine:Info:192] [2020-06-18 13:12:14.57 UTC] Fetching metadata for pool 775af3b2 from http://localhost:40195/metadata.json [cardano-wallet.pools-engine:Info:192] [2020-06-18 13:12:14.57 UTC] Successfully fetched metadata for pool 775af3b2: StakePoolMetadata {ticker = StakePoolTicker {unStakePoolTicker = "GPB"}, name = "Genesis Pool B", description = Nothing, homepage = "https://iohk.io"} [cardano-wallet.pools-engine:Info:192] [2020-06-18 13:12:14.57 UTC] Fetching metadata for pool 5a7b67c7 from http://localhost:45173/metadata.json [cardano-wallet.pools-engine:Info:192] [2020-06-18 13:12:14.57 UTC] Successfully fetched metadata for pool 5a7b67c7: StakePoolMetadata {ticker = StakePoolTicker {unStakePoolTicker = "GPC"}, name = "Genesis Pool C", description = Just "Lorem Ipsum Dolor Sit Amet.", homepage = "https://iohk.io"} [cardano-wallet.pools-engine:Info:192] [2020-06-18 13:12:14.57 UTC] Taking a little break from fetching metadata, back to it in about 0.2s ```  Co-authored-by: KtorZ <[email protected]> Co-authored-by: IOHK <[email protected]>

iohk-bors · 2020-06-18T18:38:58Z

Build failed

buildkite/cardano-wallet

KtorZ · 2020-06-18T19:52:54Z

bors r+

1776: Fetch metadata discovered on-chain from remote-peers r=KtorZ a=KtorZ # Issue Number  # Overview  - 854859c 📍 **add logic for fetching metadata from references in the database** The workflow is as follows: - Metadata are discovered on chain in pool registration certificates by the stake pool monitoring loop. Every time a new metadata URL and hash are found in a certificate, they are stored in a specific table which acts as a queue. - Then, another worker loop watches consumes items from that queue and fetch metadata corresponding to each item and stores the result in another table for long-term caching. Note: Metadata can't be updated freely since the hash of their content is stored on-chain as part of the registration certificate. New metadata will necessarily means a new certificate will show up, and will go over the whole fetching loop again. I've currently abstracted away the part that is actually fetching the metadata so that, in a first time, we can directly fetch from remote servers directly, whereas in a second time, we can replace this with a deployed centralized aggregation server like SMASH. - 3ed5c94 📍 **add and serve metadata in integration cluster pool certificates** - 25d3895 📍 **wrap raw 'Text' as 'StakePoolMetadataUrl' newtype** In order to define some necessary non-orphan instances for persitent. - 385c015 📍 **add method for storing metadata and getting next one to fetch from database** I'll add some property-based testing to make sure this works fine, in particular this is using raw sql to avoid having to do too many queries with too many things stored in-memory. # Comments  In progress, mostly untested yet. ``` [cardano-wallet.pools-engine:Info:192] [2020-06-18 13:12:14.36 UTC] Taking a little break from fetching metadata, back to it in about 0.2s [cardano-wallet.pools-engine:Info:190] [2020-06-18 13:12:14.36 UTC] Applying blocks [0.0 ... 0.0] [cardano-wallet.pools-engine:Info:190] [2020-06-18 13:12:14.36 UTC] Applying blocks [0.12 ... 2.0] [cardano-wallet.pools-engine:Info:190] [2020-06-18 13:12:14.36 UTC] Discovered stake pool registration: Registration of c7258ccc owned by [ed25519_pk1va7rf7f7p097545q8nplgwz98n7x70zk0qm8vhfx3q2apedlt7fsrttw4c] [cardano-wallet.pools-engine:Info:190] [2020-06-18 13:12:14.37 UTC] Discovered stake pool registration: Registration of 775af3b2 owned by [ed25519_pk1nl63hryqlfljan9046ztd6ejgfersnzevk6cvy2rlkr2npkmdvcqvezqk6] [cardano-wallet.pools-engine:Info:190] [2020-06-18 13:12:14.37 UTC] Discovered stake pool registration: Registration of 5a7b67c7 owned by [ed25519_pk1vyvapw37ywgugefz0cl9580272w5399m4u4d74mgrwu7st5dd20qsgg22z] [cardano-wallet.pools-engine:Info:192] [2020-06-18 13:12:14.56 UTC] Fetching metadata for pool c7258ccc from http://localhost:34159/metadata.json [cardano-wallet.pools-engine:Info:192] [2020-06-18 13:12:14.56 UTC] Successfully fetched metadata for pool c7258ccc: StakePoolMetadata {ticker = StakePoolTicker {unStakePoolTicker = "GPA"}, name = "Genesis Pool A", description = Nothing, homepage = "https://iohk.io"} [cardano-wallet.pools-engine:Info:192] [2020-06-18 13:12:14.57 UTC] Fetching metadata for pool 775af3b2 from http://localhost:40195/metadata.json [cardano-wallet.pools-engine:Info:192] [2020-06-18 13:12:14.57 UTC] Successfully fetched metadata for pool 775af3b2: StakePoolMetadata {ticker = StakePoolTicker {unStakePoolTicker = "GPB"}, name = "Genesis Pool B", description = Nothing, homepage = "https://iohk.io"} [cardano-wallet.pools-engine:Info:192] [2020-06-18 13:12:14.57 UTC] Fetching metadata for pool 5a7b67c7 from http://localhost:45173/metadata.json [cardano-wallet.pools-engine:Info:192] [2020-06-18 13:12:14.57 UTC] Successfully fetched metadata for pool 5a7b67c7: StakePoolMetadata {ticker = StakePoolTicker {unStakePoolTicker = "GPC"}, name = "Genesis Pool C", description = Just "Lorem Ipsum Dolor Sit Amet.", homepage = "https://iohk.io"} [cardano-wallet.pools-engine:Info:192] [2020-06-18 13:12:14.57 UTC] Taking a little break from fetching metadata, back to it in about 0.2s ```  Co-authored-by: KtorZ <[email protected]> Co-authored-by: IOHK <[email protected]>

iohk-bors · 2020-06-18T19:58:46Z

Canceled

KtorZ · 2020-06-18T20:01:14Z

bors r+

1776: Fetch metadata discovered on-chain from remote-peers r=KtorZ a=KtorZ # Issue Number  # Overview  - 854859c 📍 **add logic for fetching metadata from references in the database** The workflow is as follows: - Metadata are discovered on chain in pool registration certificates by the stake pool monitoring loop. Every time a new metadata URL and hash are found in a certificate, they are stored in a specific table which acts as a queue. - Then, another worker loop watches consumes items from that queue and fetch metadata corresponding to each item and stores the result in another table for long-term caching. Note: Metadata can't be updated freely since the hash of their content is stored on-chain as part of the registration certificate. New metadata will necessarily means a new certificate will show up, and will go over the whole fetching loop again. I've currently abstracted away the part that is actually fetching the metadata so that, in a first time, we can directly fetch from remote servers directly, whereas in a second time, we can replace this with a deployed centralized aggregation server like SMASH. - 3ed5c94 📍 **add and serve metadata in integration cluster pool certificates** - 25d3895 📍 **wrap raw 'Text' as 'StakePoolMetadataUrl' newtype** In order to define some necessary non-orphan instances for persitent. - 385c015 📍 **add method for storing metadata and getting next one to fetch from database** I'll add some property-based testing to make sure this works fine, in particular this is using raw sql to avoid having to do too many queries with too many things stored in-memory. # Comments  In progress, mostly untested yet. ``` [cardano-wallet.pools-engine:Info:192] [2020-06-18 13:12:14.36 UTC] Taking a little break from fetching metadata, back to it in about 0.2s [cardano-wallet.pools-engine:Info:190] [2020-06-18 13:12:14.36 UTC] Applying blocks [0.0 ... 0.0] [cardano-wallet.pools-engine:Info:190] [2020-06-18 13:12:14.36 UTC] Applying blocks [0.12 ... 2.0] [cardano-wallet.pools-engine:Info:190] [2020-06-18 13:12:14.36 UTC] Discovered stake pool registration: Registration of c7258ccc owned by [ed25519_pk1va7rf7f7p097545q8nplgwz98n7x70zk0qm8vhfx3q2apedlt7fsrttw4c] [cardano-wallet.pools-engine:Info:190] [2020-06-18 13:12:14.37 UTC] Discovered stake pool registration: Registration of 775af3b2 owned by [ed25519_pk1nl63hryqlfljan9046ztd6ejgfersnzevk6cvy2rlkr2npkmdvcqvezqk6] [cardano-wallet.pools-engine:Info:190] [2020-06-18 13:12:14.37 UTC] Discovered stake pool registration: Registration of 5a7b67c7 owned by [ed25519_pk1vyvapw37ywgugefz0cl9580272w5399m4u4d74mgrwu7st5dd20qsgg22z] [cardano-wallet.pools-engine:Info:192] [2020-06-18 13:12:14.56 UTC] Fetching metadata for pool c7258ccc from http://localhost:34159/metadata.json [cardano-wallet.pools-engine:Info:192] [2020-06-18 13:12:14.56 UTC] Successfully fetched metadata for pool c7258ccc: StakePoolMetadata {ticker = StakePoolTicker {unStakePoolTicker = "GPA"}, name = "Genesis Pool A", description = Nothing, homepage = "https://iohk.io"} [cardano-wallet.pools-engine:Info:192] [2020-06-18 13:12:14.57 UTC] Fetching metadata for pool 775af3b2 from http://localhost:40195/metadata.json [cardano-wallet.pools-engine:Info:192] [2020-06-18 13:12:14.57 UTC] Successfully fetched metadata for pool 775af3b2: StakePoolMetadata {ticker = StakePoolTicker {unStakePoolTicker = "GPB"}, name = "Genesis Pool B", description = Nothing, homepage = "https://iohk.io"} [cardano-wallet.pools-engine:Info:192] [2020-06-18 13:12:14.57 UTC] Fetching metadata for pool 5a7b67c7 from http://localhost:45173/metadata.json [cardano-wallet.pools-engine:Info:192] [2020-06-18 13:12:14.57 UTC] Successfully fetched metadata for pool 5a7b67c7: StakePoolMetadata {ticker = StakePoolTicker {unStakePoolTicker = "GPC"}, name = "Genesis Pool C", description = Just "Lorem Ipsum Dolor Sit Amet.", homepage = "https://iohk.io"} [cardano-wallet.pools-engine:Info:192] [2020-06-18 13:12:14.57 UTC] Taking a little break from fetching metadata, back to it in about 0.2s ```  Co-authored-by: KtorZ <[email protected]> Co-authored-by: IOHK <[email protected]>

iohk-bors · 2020-06-18T20:50:48Z

Build failed

ci/hydra-build:required

KtorZ · 2020-06-18T20:51:34Z

☝️

Failures:

  src/Test/Integration/Scenario/API/Shelley/Transactions.hs:357:22:
  1) API Specifications TRANS_CREATE_02 - Multiple Output Tx to different wallets
       expected: Quantity {getQuantity = 800000000000}
        but got: Quantity {getQuantity = 999999843898}

Fetching the balance right after making a transaction when the slot length is 0.2s gives actually little guarantee that the transaction doesn't already have been processed!

Sending multiple outputs to a single wallets or to many wallets has literally no differences. Addresses are addresses.

KtorZ · 2020-06-18T21:28:30Z

bors r+

iohk-bors · 2020-06-18T22:22:06Z

Build succeeded

KtorZ requested a review from Anviking June 17, 2020 17:20

KtorZ self-assigned this Jun 17, 2020

KtorZ changed the base branch from master to KtorZ/1761/extract-pool-registrations June 17, 2020 17:20

Anviking reviewed Jun 17, 2020

View reviewed changes

lib/core/src/Cardano/Pool/DB.hs Outdated Show resolved Hide resolved

KtorZ force-pushed the KtorZ/1761/extract-pool-registrations branch from fc7b95d to 2b6ea47 Compare June 17, 2020 17:30

KtorZ force-pushed the KtorZ/1762/fetch-metadata branch from 385c015 to d186224 Compare June 17, 2020 17:34

KtorZ force-pushed the KtorZ/1761/extract-pool-registrations branch from 2b6ea47 to d0386e5 Compare June 17, 2020 22:05

Anviking force-pushed the KtorZ/1761/extract-pool-registrations branch from d0386e5 to 930a3b3 Compare June 18, 2020 08:54

KtorZ force-pushed the KtorZ/1761/extract-pool-registrations branch from 930a3b3 to a52ed79 Compare June 18, 2020 09:39

KtorZ force-pushed the KtorZ/1762/fetch-metadata branch 4 times, most recently from 91a5dba to 89e2f23 Compare June 18, 2020 13:55

Anviking reviewed Jun 18, 2020

View reviewed changes

KtorZ force-pushed the KtorZ/1761/extract-pool-registrations branch from 45f9717 to 2695e52 Compare June 18, 2020 15:17

KtorZ force-pushed the KtorZ/1762/fetch-metadata branch from 1d21cf0 to 586701f Compare June 18, 2020 15:21

Base automatically changed from KtorZ/1761/extract-pool-registrations to master June 18, 2020 16:03

KtorZ marked this pull request as ready for review June 18, 2020 16:31

KtorZ added 4 commits June 18, 2020 18:52

add and serve metadata in integration cluster pool certificates

8ca2ee3

wrap raw 'Text' as 'StakePoolMetadataUrl' newtype

8adf216

In order to define some necessary non-orphan instances for persitent.

add method for storing metadata and getting next one to fetch from da…

4a925dd

…tabase I'll add some property-based testing to make sure this works fine, in particular this is using raw sql to avoid having to do too many queries with too many things stored in-memory.

KtorZ force-pushed the KtorZ/1762/fetch-metadata branch 2 times, most recently from 4aecaaf to 2a37581 Compare June 18, 2020 17:25

Anviking approved these changes Jun 18, 2020

View reviewed changes

KtorZ added 4 commits June 18, 2020 19:51

define 'Crypto.Hash.Utils' to avoid re-defining the same hashing func…

89ce0c8

…tions everywhere

write simple albeit defensive client fetching metadata from remote peers

dbd06e4

make prettier (and fix) log output for stake pool fetch pause

cff1c31

replace hard-coded active slot coefficient value with actual value fr…

a0a889b

…om genesis

KtorZ force-pushed the KtorZ/1762/fetch-metadata branch from 2a37581 to a58a193 Compare June 18, 2020 17:51

KtorZ added 2 commits June 18, 2020 20:04

property-test fetching metadata references from database

6784e5a

implement putPoolMetadata and unfetchedPoolMetadata for the database …

f48815b

…model

KtorZ force-pushed the KtorZ/1762/fetch-metadata branch from a58a193 to c18da60 Compare June 18, 2020 18:11

KtorZ added the ADDING FEATURE Mark a PR as adding a new feature, for auto-generated CHANGELOG label Jun 18, 2020

KtorZ and others added 4 commits June 18, 2020 23:28

remove redundant dependencies

dc49cae

fix race-condition in integration scenario

803ef44

Fetching the balance right after making a transaction when the slot length is 0.2s gives actually little guarantee that the transaction doesn't already have been processed!

remove redundant integration tests

d0bf34b

Sending multiple outputs to a single wallets or to many wallets has literally no differences. Addresses are addresses.

Regenerate nix

331d9f2

KtorZ force-pushed the KtorZ/1762/fetch-metadata branch from 604dc39 to 331d9f2 Compare June 18, 2020 21:28

iohk-bors bot merged commit 3010d4e into master Jun 18, 2020

iohk-bors bot deleted the KtorZ/1762/fetch-metadata branch June 18, 2020 22:22

KtorZ mentioned this pull request Jun 22, 2020

Collect stake pool metadata from remote servers #1762

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fetch metadata discovered on-chain from remote-peers #1776

Fetch metadata discovered on-chain from remote-peers #1776

KtorZ commented Jun 17, 2020 •

edited

Loading

Anviking Jun 18, 2020 •

edited

Loading

KtorZ Jun 18, 2020

Anviking Jun 18, 2020

Anviking Jun 18, 2020

KtorZ Jun 18, 2020

Anviking Jun 18, 2020

Anviking Jun 18, 2020

KtorZ Jun 18, 2020

Anviking Jun 18, 2020

KtorZ Jun 18, 2020

KtorZ commented Jun 18, 2020

iohk-bors bot commented Jun 18, 2020

KtorZ commented Jun 18, 2020

iohk-bors bot commented Jun 18, 2020

KtorZ commented Jun 18, 2020

iohk-bors bot commented Jun 18, 2020

KtorZ commented Jun 18, 2020

KtorZ commented Jun 18, 2020

iohk-bors bot commented Jun 18, 2020

	parseJSON = withObject "StakePoolMetadta" $ \obj -> do
	parseJSON = withObject "StakePoolMetadata" $ \obj -> do

Fetch metadata discovered on-chain from remote-peers #1776

Fetch metadata discovered on-chain from remote-peers #1776

Conversation

KtorZ commented Jun 17, 2020 • edited Loading

Issue Number

Overview

Comments

Anviking Jun 18, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

KtorZ commented Jun 18, 2020

iohk-bors bot commented Jun 18, 2020

Build failed

KtorZ commented Jun 18, 2020

iohk-bors bot commented Jun 18, 2020

Canceled

KtorZ commented Jun 18, 2020

iohk-bors bot commented Jun 18, 2020

Build failed

KtorZ commented Jun 18, 2020

KtorZ commented Jun 18, 2020

iohk-bors bot commented Jun 18, 2020

Build succeeded

KtorZ commented Jun 17, 2020 •

edited

Loading

Anviking Jun 18, 2020 •

edited

Loading