Adds transaction metrics for accounts & data loaded/stored

[brooksprumo]

Problem

While processing transactions, we track metrics related to how long it takes to load and store accounts, but not the number of accounts¹ nor the amount of account data.

Summary of Changes

Add metrics for the number of accounts and amount of account data loaded and stored during transaction processing.

Here's an example of the new metrics:

number of accounts

account data

Footnotes

1. Caveat: We do track the number of accounts that are passed to message processing, "execute_details_total_account_count", which matches the number of accounts loaded. So this PR adds a currently-redundant datapoint for number of loaded accounts. I can remove that if desired. Is there something similar on the store-side? I.e. a number-of-modified-accounts-to-be-stored? ↩

[codecov]

Codecov Report

Attention: 3 lines in your changes are missing coverage. Please review.

Comparison is base (c6c2340) 81.6% compared to head (001acc4) 81.6%.

Additional details and impacted files

@@           Coverage Diff           @@
##           master   #34718   +/-   ##
=======================================
  Coverage    81.6%    81.6%           
=======================================
  Files         830      830           
  Lines      224837   224866   +29     
=======================================
+ Hits       183505   183564   +59     
+ Misses      41332    41302   -30     

[alessandrod]

nit: I'd prefer to avoid the extra loop and instead compute the value in load_accounts

[brooksprumo]

Done. Fixed in 6948ff3.

[brooksprumo]

And changed in 006ff6c.

[jeffwashington]

there is a fn maybe in append vec or accounts that tells you the total bytes, including owner, etc. This is only summing data len(), which is different than bytes stored.

[brooksprumo]

Yeah, that's a good point. I was originally intending to track only the account data, but maybe tracking all the loaded/stored bytes would be better? Or maybe do both? Wdyt?

[brooksprumo]

Thinking about this more, I actually do want the account data size for the datapoint itself. I've renamed the datapoint to hopefully be more clear that this is the account data and not all the bytes.

We'll also now have the number of accounts stored, so the metadata bytes could be added to someone's query, if they'd like.

[brooksprumo]

Rebased to address merge conflicts.

[jeffwashington]

we iterate these and sum data len here and accumulate a stat:

solana/accounts-db/src/accounts_db.rs

Lines 8459 to 8461 in 897adb2

	self.stats
	.store_total_data
	.fetch_add(total_data as u64, Ordering::Relaxed);

[brooksprumo]

Ah nice. I'd like to get these stats per slot though. IIUC, this stat will be per second.

Also maybe getting the stored amount per slot is not actually needed? It does feel nice to exactly correlate any replay load/store time increases to the number of accounts or the amount of data read/written though.

[willhickey]

This repository is no longer in use. Please re-open this pull request in the agave repo: https://github.com/anza-xyz/agave