Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Call removeRetiredPools from monitorStakePools. #2057

Merged
merged 21 commits into from
Sep 1, 2020

Conversation

jonathanknowles
Copy link
Contributor

@jonathanknowles jonathanknowles commented Aug 24, 2020

Issue Number

#2019

Overview

PR #2057 provided the removeRetiredPools function which, when given an epoch, removes pools from the database that retired on or before that epoch.

This PR adjusts monitorStakePools to actually call removeRetiredPools.

Frequency of Garbage Collection

Since garbage collection has a runtime cost associated with it, we want to avoid doing it more than necessary, even if the cost is relatively small.

Thus, we only perform garbage collection:

  • on demand, just before we record a new pool registration or retirement certificate in the database.
  • at most once per epoch.

Logging Examples

A single garbage collection invocation (at most once per epoch)

Click to view log sample
[cardano-wallet.pools-engine:Debug:31] [2020-08-31 05:42:24.77 UTC] Performing garbage collection of retired stake pools. Currently in epoch 214. Removing all pools that retired in or before epoch 212.
[cardano-wallet.pools-db:Debug:31] [2020-08-31 05:42:24.77 UTC] Removing pools that retired in or before epoch 212: start
[cardano-wallet.pools-db:Debug:31] [2020-08-31 05:42:24.78 UTC] Removing the following retired pools:
Pool 215622e6 with retirement epoch 212
Pool 5687ecfe with retirement epoch 212
Pool 69d158eb with retirement epoch 212
Pool 7bacd597 with retirement epoch 212
Pool 8e681030 with retirement epoch 212
Pool a6d582d8 with retirement epoch 212
Pool bccf5ce0 with retirement epoch 212
Pool c0b86026 with retirement epoch 212
Pool cfe83240 with retirement epoch 212
Pool f5863386 with retirement epoch 212
[cardano-wallet.pools-db:Notice:31] [2020-08-31 05:42:24.78 UTC] Removing the following pool from the database: 215622e669c0f0c09828ec4100a4c689b1f9c148f542b0192b6ed00d.
[cardano-wallet.pools-db:Notice:31] [2020-08-31 05:42:24.80 UTC] Removing the following pool from the database: 5687ecfeaf19ed1a7ec20a9332ff80c8ad6fc5ace97cfb7276d256a2.
[cardano-wallet.pools-db:Notice:31] [2020-08-31 05:42:24.82 UTC] Removing the following pool from the database: 69d158eb61cbbeff3ae564768eef7c0b52ba8bd33b2301eacf62a24d.
[cardano-wallet.pools-db:Notice:31] [2020-08-31 05:42:24.84 UTC] Removing the following pool from the database: 7bacd59728317961e6ffb20f40f74706954a5341d1159f7e2be32a68.
[cardano-wallet.pools-db:Notice:31] [2020-08-31 05:42:24.85 UTC] Removing the following pool from the database: 8e68103060b32595d4eed0a7749795b155d790b95e0615f9f5618b9f.
[cardano-wallet.pools-db:Notice:31] [2020-08-31 05:42:24.87 UTC] Removing the following pool from the database: a6d582d86cafd861984ec33f3c4bb7ab1e183d0719abc92351dd2dd7.
[cardano-wallet.pools-db:Notice:31] [2020-08-31 05:42:24.89 UTC] Removing the following pool from the database: bccf5ce0d9b3436cd1616686c98f4584b43760704c9585f1f208ffae.
[cardano-wallet.pools-db:Notice:31] [2020-08-31 05:42:24.90 UTC] Removing the following pool from the database: c0b86026e65261a0698e789fc2e0b1cf36c85693dbf1cd516067400d.
[cardano-wallet.pools-db:Notice:31] [2020-08-31 05:42:24.92 UTC] Removing the following pool from the database: cfe83240dd1eb92bac3d739b2550be334f4bb7e4fc11d7f5faeb8769.
[cardano-wallet.pools-db:Notice:31] [2020-08-31 05:42:24.94 UTC] Removing the following pool from the database: f58633867bea26335d12aa95a8b3b1dee5b6994ffd4bac5348375220.
[cardano-wallet.pools-db:Debug:31] [2020-08-31 05:42:24.95 UTC] Removing pools that retired in or before epoch 212: finish

Lifecycle of a single pool

Click to view log sample
[cardano-wallet.pools-engine:Info:37] [2020-08-27 15:35:54.31 UTC] Discovered stake pool registration: Registration of 00008d6a owned by [ed25519_pk1lrrgks5hnpxqqpyhjxedk7u5nxp0ehp9tm9ksx4a7zepzt6ydnw]
...
[cardano-wallet.pools-engine:Info:37] [2020-08-27 15:35:54.85 UTC] Discovered stake pool retirement: Pool 00008d6a with retirement epoch 209
...
[cardano-wallet.pools-engine:Debug:37] [2020-08-27 15:36:16.07 UTC] Performing garbage collection of retired stake pools. Currently in epoch 211. Removing all pools that retired in or before epoch 209.
[cardano-wallet.pools-db:Notice:37] [2020-08-27 15:36:16.08 UTC] Removing the following pool from the database: 00008d6abd6bf0dafbc7f64b68267f5716e1992d0e845eb22d05065b.

@jonathanknowles jonathanknowles self-assigned this Aug 24, 2020
@jonathanknowles jonathanknowles added the IMPROVEMENT Mark a PR as an improvement, for auto-generated CHANGELOG label Aug 24, 2020
Base automatically changed from jonathanknowles/remove-retired-pools to master August 24, 2020 11:19
@jonathanknowles jonathanknowles force-pushed the jonathanknowles/call-remove-retired-pools branch from 617ce4b to 5296008 Compare August 24, 2020 11:20
iohk-bors bot added a commit that referenced this pull request Aug 24, 2020
@iohk-bors

This comment has been minimized.

@jonathanknowles jonathanknowles force-pushed the jonathanknowles/call-remove-retired-pools branch 4 times, most recently from 42a8250 to 0c5e82d Compare August 25, 2020 04:02
@jonathanknowles jonathanknowles marked this pull request as ready for review August 25, 2020 04:09
@jonathanknowles jonathanknowles force-pushed the jonathanknowles/call-remove-retired-pools branch from b1e4d15 to 812b0c6 Compare August 25, 2020 04:24
@jonathanknowles
Copy link
Contributor Author

bors try

iohk-bors bot added a commit that referenced this pull request Aug 25, 2020
@jonathanknowles jonathanknowles changed the title Call removeRetiredPools from within monitorStakePools. Call removeRetiredPools from monitorStakePools. Aug 25, 2020
@iohk-bors

This comment has been minimized.

jonathanknowles added a commit that referenced this pull request Aug 25, 2020
jonathanknowles added a commit that referenced this pull request Aug 25, 2020
@jonathanknowles jonathanknowles force-pushed the jonathanknowles/call-remove-retired-pools branch from bfb0609 to 78e8e04 Compare August 25, 2020 08:44
jonathanknowles added a commit that referenced this pull request Aug 25, 2020
@jonathanknowles jonathanknowles force-pushed the jonathanknowles/call-remove-retired-pools branch from 78e8e04 to e0d5eb7 Compare August 25, 2020 08:46
@jonathanknowles jonathanknowles requested review from Anviking and KtorZ and removed request for Anviking August 25, 2020 09:29
Copy link
Member

@Anviking Anviking left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

By only performing garbage collection when we actually have new pool
certificates to add, we avoid paying the cost of garbage collection in
every single slot.
When garbage collecting pools.
We don't need to see any informational messages when there are no pools
to garbage collect.

In response to review feedback:

https://github.com/input-output-hk/cardano-wallet/pull/2057/files#r476141310
The nested function definitions did not depend on the outer function, so
it is safe to move them up one level.
In this rationale, we explain why we only garbage collect pools that
retired at least two epochs before the current epoch.

In response to review feedback:

#2057 (comment)
Justification:

Some `DBLayer` functions are defined in terms of other `DBLayer`
functions, such as `readPoolLifeCycleStatus`, which is defined in terms
of `readPoolRegistration` and `readPoolRetirement`.

Since mutual recursion between functions defined as fields of a record
is somewhat awkward, we currently define auxilliary functions for each
of the functions that we need to reuse, and then refer to these
auxilliary functions when building the record. However, this creates a
layer of indirection and duplication.

By using `RecordWildCards`, we can instead build a `DBLayer` object in
terms of ordinary functions defined in a `where` clause. Since ordinary
functions can refer to one another using mutual recursion, we no longer
need auxilliary functons, saving a layer of indirection.
With the use of `RecordWildCards`, these auxilliary functions are no
longer necessary, as functions in a where clause can refer to one
another directly.
Nesting a call to `atomically` will result in the nested call blocking
indefinitely.

This fixes:

#2057 (comment)
#2057 (comment)

This change also removes bracketing for the `MsgStakePoolGarbageCollection`
constructor of `StakePoolLog`, as there are no longer any child log entries
belonging to the pools engine.

(Other log entries pertaining to garbage collection are emitted by the pool
DB component, as a result of calling the `removeRetiredPools` DB function.)
@jonathanknowles jonathanknowles force-pushed the jonathanknowles/call-remove-retired-pools branch from 6ae4115 to 5daa926 Compare September 1, 2020 01:45
@jonathanknowles
Copy link
Contributor Author

bors r+

@iohk-bors
Copy link
Contributor

iohk-bors bot commented Sep 1, 2020

Build succeeded

@iohk-bors iohk-bors bot closed this in d00e6b9 Sep 1, 2020
@iohk-bors iohk-bors bot merged commit 455ca66 into master Sep 1, 2020
@iohk-bors iohk-bors bot deleted the jonathanknowles/call-remove-retired-pools branch September 1, 2020 02:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
IMPROVEMENT Mark a PR as an improvement, for auto-generated CHANGELOG
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants