Add extended history tracking for select accounts

[christophersanborn]

Addresses #2258 - "Extended operation history for accounts selected by ID or by registrar".

Adds the following options to account_history plugin:

  --extended-max-ops-per-account arg    Maximum number of operations to keep 
                                        for accounts for which extended history is kept
  --extended-history-by-account arg     Track longer history for these accounts
                                        (may specify multiple times)
  --extended-history-by-registrar arg   Track longer history for accounts with 
                                        this registrar (may specify multiple times)

This PR may also add a partial, if imperfect, solution to #2117 "Track accounts using wildcards".

Note by @abitmore: check this comment for limitation of current implementation: #2259 (review)

[christophersanborn]

Note: code compiles and starts up, but I haven't done a full replay yet to confirm that it is indeed tracking extended history for my accounts. So this code is still in "testing".

@abitmore — I assume this will need to be rebased onto another branch — let me know if develop, hardfork, or elsewhere.

[abitmore]

Please rebase onto develop branch. Thanks.

[christophersanborn]

@abitmore wrote:

Please rebase onto develop branch. Thanks.

Done.

Also my node finished replay and I confirmed it is tracking extended history for the named accounts/registrars and regular (short) history for all other accounts. Looks to be working. (Although my hopes for extended history in the Reference GUI did not pan out — but this seems to be a limitation of the UI, not the API node. Uptick was able to pull all the extended history.)

[abitmore]

Thanks. I'll review soon.

[abitmore]

Thanks for the efforts.

The implementation would work for typical use cases. However,

• it does not work if _max_ops_per_account is set to 0 but extended_* is set, especially if the node operator mainly wants to track by registrars;
• it does not work if _tracked_account is not empty and extended_* is not within _tracked_accounts;
• performance can be improved (IMHO) quite a bit as mentioned in the in-line comments;
• tracking by registrar stops when the account upgrades to LTM (The extended-history-by-registrar option in account history plugin does not track LTM correctly #2656).

[abitmore]

Querying DB once on every operation is a bit inefficient. Perhaps better create a new index in the plugin to save the tracked accounts. Why a new index? Because the list is dynamic and grows over time.

[abitmore]

Although the account_index is big, the complexity of querying it by ID is linear, so the performance hit is probably not significant. Best if we can have some data to show the differences E.G. time spent for replaying with the old code v.s. the new code.

[abitmore]

I will decide later whether to merge it as is.

[BhuzOr]

What about something more flexible and powerful like this?

--extended-history-by-account account, max_ops, whitelisted_ops, blacklisted_ops

may specify multiple times

--extended-history-by-account account1, max_ops_1, whitelisted_ops_1, blacklisted_ops_1
--extended-history-by-account account2, max_ops_2, whitelisted_ops_2, blacklisted_ops_2

or as a list of lists

--extended-history-by-account [[account1, max_ops_1, whitelisted_ops_1, blacklisted_ops_1], [account2, max_ops_2, whitelisted_ops_2, blacklisted_ops_2]]

You have specific max_ops per single account/registrar instead of a single global value for everything.
You can also apply a filter, specifying what type of ops to keep OR what type of ops to discard, for that specific account/registrar

[abitmore]

Approving and merging so far. We can improve later.