Extend restoration benchmarks to include large random wallets. #2081
Conversation
KtorZ
added
the
RESOLVING ISSUE
| -- seed. | ||
| -- | ||
| -- The first argument is expected to be a ratio (between 0 and 1) of addresses | ||
| -- we ought to simply recognize as ours. So, giving .5 means that 50% of the |
There was a problem hiding this comment.
Seems outdated. Should be ratio (between 0% and 100%) and giving @50 means 50% of the I presume?
There was a problem hiding this comment.
Right, I had it as an argument initially and switched to a type-level parameter later on.
giving @50 means 50% of the I presume?
Indeed.
| -- For benchmarking and testing arbitrary large random wallets. | ||
|
|
||
| -- | An "unsound" alternative that can be used for benchmarking and stress | ||
| -- testing. It re-uses the same underlying structure as the `RndState` but |
There was a problem hiding this comment.
same underlying structure
Is there a main difference, say in complexity, between RndAnyState and AnyAddressState?
There was a problem hiding this comment.
Would this be true?
RndAnyState, like RndState, grow with the number of addresses used. This is unlike SeqState/AnyAddressState, where which has a bounded gap of user addresses it keeps in memory
If not, something like it, would be cool to understand what makes RndAnyState indicative of performance of random wallets, and why it's not enough to have AnyAddressState.
There was a problem hiding this comment.
Re my earlier comment:
Ah, I'm blind 😅
innerState :: RndState network"same underlying structure" makes sense then
There was a problem hiding this comment.
:) ... It still stores the UTxO and transaction history, but the AnyState do not store the address space. BUT, we've observed some major slowness when it comes to storing addresses, so I find it quite important to also include that in the benchmark.
I think that we could have a similar SeqAnyState which also include the seq state, and get rid of the AnyState altogether.
lib/shelley/bench/Restore.hs
Outdated
| socketFile | ||
| np | ||
| vData | ||
| "1-percent-naked.timelog" |
There was a problem hiding this comment.
Maybe just 1-percent.timelog would be easier
|
|
||
| , bench ("restore " <> network <> " 2% ownership") |
There was a problem hiding this comment.
Removing because it was too slow, I presume?
Maybe it's worth adding 0.1%, or 0.5%, to still have more data points than 0% and 1%.
(Useful, I imagine, if you'd ever want to do some curve fitting on the data)
There was a problem hiding this comment.
Removing because it was too slow, I presume?
Correct.
Maybe it's worth adding 0.1%, or 0.5%, to still have more data points than 0% and 1%.
Good idea 👍
| -- For benchmarking and testing arbitrary large random wallets. | ||
|
|
||
| -- | An "unsound" alternative that can be used for benchmarking and stress | ||
| -- testing. It re-uses the same underlying structure as the `RndState` but |
There was a problem hiding this comment.
Re my earlier comment:
Ah, I'm blind 😅
innerState :: RndState network"same underlying structure" makes sense then
This wallet is very analogous to the existing any% wallet scheme we designed a while ago, with a subtle difference: it is built __on top of__ the 'RndState' and, as a result, does perform the same database operation and addresses management as the standard random wallets. So the benchmark results obtained from this are much closer to what an actual random wallet of the same size would look like.
This is actually the implementation we ought to test in these benchmark.
KtorZ
force-pushed
the
KtorZ/2032/extend-restoration-benchmarks
branch
from
f0a4bf9 to
580bd96
Compare
|
@Anviking added .1% and .5% benchmarks + fix comment to correctly mention the type parameter. |
| (False, _) | crc32 bytes < p -> | ||
| let | ||
| (path, gen') = findUnusedPath | ||
| (gen inner) (accountIndex inner) (unavailablePaths inner) |
There was a problem hiding this comment.
Couldn't this extra findUnusedPath call, compared to the inner state implementation, affect the results?
There was a problem hiding this comment.
A bit yes. We don't benchmark this function independently but I believe that even on large wallets with 100K+ addresses, finding an unused path in a space of 2^31 is quite fast.
There was a problem hiding this comment.
So I'd expect it to be quite negligible.
|
bors r+ |
Build succeeded |
Issue Number
#2032
Overview
I have defined a new special wallet scheme analogous to the any% scheme used until now, but built on top of the
RndState, so that we still get to store and retrieve the address space too.I have removed the 2% case and "replaced" it with a 1% case on large random wallets.
Comments
Tested on the testnet, which works fine. The reason to add this one is to be able to create arbitrarily large random wallets that "make sense", and then measure the time taken for other specific operations such as listing addresses or estimating fees (which I'll do as a separate PR).