blockstore: Allocate rocksdb keys more efficiently #3603
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem
The Blockstore stores different data types in different columns. More so, the key type might vary column to column. For example, the key type for a
SlotMeta
is just aSlot
as shown here:agave/ledger/src/blockstore.rs
Lines 497 to 500 in 7629a15
Under the hood, rocksdb just wants a
&[u8]
for the key. So, we have a function for turning the type (Slot
from our example) into aVec<u8>
:agave/ledger/src/blockstore_db.rs
Line 772 in 7629a15
Thus, we'll perform an extra allocation to transform the key into a byte array for every get/put operation.
Summary of Changes
LedgerColumn
KEY_LEN
field toColumn
traitC::key()
(which allocates a Vec) to the following form (which will get allocated on the stack sinceK
is known at compile time):MultiGetBytes
helper to only perform oneVec
allocation formulti_get()
calls, instead of oneVec
per key +Vec
to hold all keysPerformance
This item isn't the worst offender in the flamegraphs. But after adding in a temporary counter, I see that tip-of-master is calling
C::key()
~18.5k times per second on mnb. Each of those calls is allocating aVec
, so shaving all those heap allocations is a win