feat: add metrics: balance per currency, number of unspent outputs, number of addresses #1666
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
InoMurko
reviewed
Jul 24, 2020
| end | ||
|
|
||
| defp collect_stats(db) do | ||
| {:ok, utxos} = db.utxos() |
There was a problem hiding this comment.
isn't this like the worst query you could make? pull millions of records into memory?
There was a problem hiding this comment.
Yup! Without revamping the data structure, we can chunk-calculate this, but that means a full table scan for each query, e.g. one to group by currency, another one to group by address, etc. and that's read-heavy on rocksdb which is even more severe than being memory-intensive.
And we need utxoset observability so 🤷♂️
There was a problem hiding this comment.
Actually hang on, lemme try create OMG.DB.RocksDB.reduce() that processes each utxo while it iterates through the full utxoset.
5 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Close #1148
Overview
Add metrics for childchain's balances, number of unspent outputs and addresses with unspent outputs.
Changes
total_unspent_outputs: The total number of unspent outputs being tracked in the childchain servicetotal_unspent_addresses: The total number of addresses holding at least 1 unspent outputbalance: Same definition. Fixed where the balance was calculating only from the recent cacheunique_users: Replaced bytotal_unspent_addressesNote that I originally wanted to avoid or optimize utxos calling but it requires lower level opimization and the current DB can handle the load for the foreseeable future (see #1667 for some benchmark).
With this PR we get the utxos db observability and can monitor when the metrics become concerning so I think it's worth the extra call.
Testing
Follow these steps:
DD_DISABLED: falseandDD_API_KEYat least on the childchainmake docker-buildcd priv/cabbage && make start_daemon_services-2