Skip to content

Add extended history tracking for select accounts #2259

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Sep 17, 2020
Merged

Add extended history tracking for select accounts #2259

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 44 additions & 6 deletions libraries/plugins/account_history/account_history_plugin.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,9 +64,12 @@ class account_history_plugin_impl

account_history_plugin& _self;
flat_set<account_id_type> _tracked_accounts;
flat_set<account_id_type> _extended_history_accounts;
flat_set<account_id_type> _extended_history_registrars;
bool _partial_operations = false;
primary_index< operation_history_index >* _oho_index;
uint64_t _max_ops_per_account = -1;
uint64_t _extended_max_ops_per_account = -1;
private:
/** add one history record, then check and remove the earliest history record */
void add_account_history( const account_id_type account_id, const operation_history_id_type op_id );
Expand Down Expand Up @@ -212,9 +215,25 @@ void account_history_plugin_impl::add_account_history( const account_id_type acc
obj.most_recent_op = ath.id;
obj.total_ops = ath.sequence;
});
// remove the earliest account history entry if too many
// _max_ops_per_account is guaranteed to be non-zero outside
if( stats_obj.total_ops - stats_obj.removed_ops > _max_ops_per_account )
// Amount of history to keep depends on if account is in the "extended history" list
bool extended_hist = false;
for ( auto eh_account_id : _extended_history_accounts ) {
extended_hist |= (account_id == eh_account_id);
}
if ( _extended_history_registrars.size() > 0 ) {
const account_id_type registrar_id = account_id(db).registrar;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Querying DB once on every operation is a bit inefficient. Perhaps better create a new index in the plugin to save the tracked accounts. Why a new index? Because the list is dynamic and grows over time.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Although the account_index is big, the complexity of querying it by ID is linear, so the performance hit is probably not significant. Best if we can have some data to show the differences E.G. time spent for replaying with the old code v.s. the new code.

for ( auto eh_registrar_id : _extended_history_registrars ) {
extended_hist |= (registrar_id == eh_registrar_id);
}
}
// _max_ops_per_account is guaranteed to be non-zero outside; max_ops_to_keep
// will likewise be non-zero, and also non-negative (it is unsigned).
auto max_ops_to_keep = _max_ops_per_account;
if (extended_hist && _extended_max_ops_per_account > max_ops_to_keep) {
max_ops_to_keep = _extended_max_ops_per_account;
}
// Remove the earliest account history entry if too many.
if( stats_obj.total_ops - stats_obj.removed_ops > max_ops_to_keep )
{
// look for the earliest entry
const auto& his_idx = db.get_index_type<account_transaction_history_index>();
Expand Down Expand Up @@ -283,9 +302,20 @@ void account_history_plugin::plugin_set_program_options(
)
{
cli.add_options()
("track-account", boost::program_options::value<std::vector<std::string>>()->composing()->multitoken(), "Account ID to track history for (may specify multiple times)")
("partial-operations", boost::program_options::value<bool>(), "Keep only those operations in memory that are related to account history tracking")
("max-ops-per-account", boost::program_options::value<uint64_t>(), "Maximum number of operations per account will be kept in memory")
("track-account", boost::program_options::value<std::vector<std::string>>()->composing()->multitoken(),
"Account ID to track history for (may specify multiple times; if unset will track all accounts)")
("partial-operations", boost::program_options::value<bool>(),
"Keep only those operations in memory that are related to account history tracking")
("max-ops-per-account", boost::program_options::value<uint64_t>(),
"Maximum number of operations per account that will be kept in memory")
("extended-max-ops-per-account", boost::program_options::value<uint64_t>(),
"Maximum number of operations to keep for accounts for which extended history is kept")
("extended-history-by-account",
boost::program_options::value<std::vector<std::string>>()->composing()->multitoken(),
"Track longer history for these accounts (may specify multiple times)")
("extended-history-by-registrar",
boost::program_options::value<std::vector<std::string>>()->composing()->multitoken(),
"Track longer history for accounts with this registrar (may specify multiple times)")
;
cfg.add(cli);
}
Expand All @@ -303,6 +333,14 @@ void account_history_plugin::plugin_initialize(const boost::program_options::var
if (options.count("max-ops-per-account")) {
my->_max_ops_per_account = options["max-ops-per-account"].as<uint64_t>();
}
if (options.count("extended-max-ops-per-account")) {
auto emopa = options["extended-max-ops-per-account"].as<uint64_t>();
my->_extended_max_ops_per_account = (emopa > my->_max_ops_per_account) ? emopa : my->_max_ops_per_account;
}
LOAD_VALUE_SET(options, "extended-history-by-account", my->_extended_history_accounts,
graphene::chain::account_id_type);
LOAD_VALUE_SET(options, "extended-history-by-registrar", my->_extended_history_registrars,
graphene::chain::account_id_type);
}

void account_history_plugin::plugin_startup()
Expand Down