Skip to content

Bonsai archive seperate column families #10058

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
jframe merged 21 commits into from
Mar 19, 2026
Merged

Bonsai archive seperate column families #10058

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
e0f2876
feat: separate archive CFs into FREEZER and ARCHIVE variants
jframe Mar 12, 2026
b525974
refactor: update archive methods to use FREEZER CFs
jframe Mar 12, 2026
9600549
refactor: update BonsaiArchiveFlatDbStrategy fallback reads to use FR…
jframe Mar 12, 2026
a3a6bef
test: update ArchiverTests to use FREEZER CFs
jframe Mar 12, 2026
9c1c334
feat: update BonsaiArchiveFlatDbStrategy to write migration data to A…
jframe Mar 12, 2026
3793d4b
test: update archive migration tests to use ARCHIVE CFs
jframe Mar 12, 2026
72d8202
move the freezer archive column families to after the archive column …
jframe Mar 17, 2026
8a86697
Update remaining uses of to use archive columns in BonsaiArchiveFlatD…
jframe Mar 17, 2026
5847f5b
Change main bonsai flatDB segments to archive segments for archiving
jframe Mar 17, 2026
4403f50
Move archive tests to the correct package to match BonsaiArchiver
jframe Mar 17, 2026
fa86737
Fix BonsaiArchiver and BonsaiWorldStateKeyValueStorage tests for ARCH…
jframe Mar 17, 2026
54a77aa
spotless
jframe Mar 17, 2026
d9ca67e
Remove duplicate clear archive segment code
jframe Mar 17, 2026
80b0e6c
add final on method arg
jframe Mar 17, 2026
664f0fe
Add freezer segment cleanup tests for BonsaiWorldStateKeyValueStorage
jframe Mar 17, 2026
9f37abb
Add freezer segment cleanup tests to BonsaiArchiveFlatDbStrategyTest
jframe Mar 17, 2026
452510e
Clean up unnecessary comments in freezer cleanup tests
jframe Mar 17, 2026
89c7669
Remove archive clear tests from BonsaiWorldStateKeyValueStorageTest
jframe Mar 17, 2026
0770752
remove unused method
jframe Mar 17, 2026
6508a3b
Changelog entry
jframe Mar 19, 2026
fb57211
Merge remote-tracking branch 'upstream/main' into bonsai_archive_sepe…
jframe Mar 19, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
- Deprecated `--min-block-occupancy-ratio` for removal and make it noop. That option, that is ignored on PoS networks, is related to the deprecated PoW, and allowed to broadcast a mined block as soon as it reached a satisfying fill threshold. The option is still recognized, but it has no effect and will be completely removed in a future release. [#10036](https://github.com/besu-eth/besu/pull/10036)
- Plugin API
- Removed `TransactionSelectionResult.BLOCK_OCCUPANCY_ABOVE_THRESHOLD`, in general it could be replaced with `BLOCK_FULL`
- Experimental Bonsai Archive column families have changed to improve performance during bonsai to archive migration. If you are using the Bonsai archive you will need to do a full resync [#10058](https://github.com/besu-eth/besu/pull/10058/changes)

### Upcoming Breaking Changes
- RPC changes to enhance compatibility with other ELs
Expand Down
12 changes: 12 additions & 0 deletions ...c/main/java/org/hyperledger/besu/ethereum/storage/keyvalue/KeyValueSegmentIdentifier.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,18 @@ public enum KeyValueSegmentIdentifier implements SegmentIdentifier {
true,
false,
true),
ACCOUNT_INFO_STATE_FREEZER(
"ACCOUNT_INFO_STATE_FREEZER".getBytes(StandardCharsets.UTF_8),
EnumSet.of(X_BONSAI_ARCHIVE),
true,
false,
true),
ACCOUNT_STORAGE_FREEZER(
"ACCOUNT_STORAGE_FREEZER".getBytes(StandardCharsets.UTF_8),
EnumSet.of(X_BONSAI_ARCHIVE),
true,
false,
true),
VARIABLES(new byte[] {11}), // formerly GOQUORUM_PRIVATE_WORLD_STATE

// previously supported GoQuorum private states
Expand Down
65 changes: 47 additions & 18 deletions ...rledger/besu/ethereum/trie/pathbased/bonsai/storage/flat/BonsaiArchiveFlatDbStrategy.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,10 @@
*/
package org.hyperledger.besu.ethereum.trie.pathbased.bonsai.storage.flat;

import static org.hyperledger.besu.ethereum.storage.keyvalue.KeyValueSegmentIdentifier.ACCOUNT_INFO_STATE;
import static org.hyperledger.besu.ethereum.storage.keyvalue.KeyValueSegmentIdentifier.ACCOUNT_INFO_STATE_ARCHIVE;
import static org.hyperledger.besu.ethereum.storage.keyvalue.KeyValueSegmentIdentifier.ACCOUNT_INFO_STATE_FREEZER;
import static org.hyperledger.besu.ethereum.storage.keyvalue.KeyValueSegmentIdentifier.ACCOUNT_STORAGE_ARCHIVE;
import static org.hyperledger.besu.ethereum.storage.keyvalue.KeyValueSegmentIdentifier.ACCOUNT_STORAGE_STORAGE;
import static org.hyperledger.besu.ethereum.storage.keyvalue.KeyValueSegmentIdentifier.ACCOUNT_STORAGE_FREEZER;
import static org.hyperledger.besu.ethereum.storage.keyvalue.KeyValueSegmentIdentifier.TRIE_BRANCH_STORAGE;
import static org.hyperledger.besu.ethereum.trie.pathbased.common.storage.PathBasedWorldStateKeyValueStorage.WORLD_BLOCK_NUMBER_KEY;

Expand Down Expand Up @@ -133,7 +133,7 @@ public Optional<Bytes> getFlatAccount(
// Find the nearest account state for this address and block context
Optional<SegmentedKeyValueStorage.NearestKeyValue> nearestAccount =
storage
.getNearestBefore(ACCOUNT_INFO_STATE, keyNearest)
.getNearestBefore(ACCOUNT_INFO_STATE_ARCHIVE, keyNearest)
.filter(
found ->
accountHash.getBytes().commonPrefixLength(found.key())
Expand All @@ -143,7 +143,7 @@ public Optional<Bytes> getFlatAccount(
if (nearestAccount.isEmpty()) {
accountFound =
storage
.getNearestBefore(ACCOUNT_INFO_STATE_ARCHIVE, keyNearest)
.getNearestBefore(ACCOUNT_INFO_STATE_FREEZER, keyNearest)
.filter(
found ->
accountHash.getBytes().commonPrefixLength(found.key())
Expand Down Expand Up @@ -180,7 +180,7 @@ protected Stream<Pair<Bytes32, Bytes>> accountsToPairStream(
final Stream<Pair<Bytes32, Bytes>> stream =
storage
.streamFromKey(
ACCOUNT_INFO_STATE,
ACCOUNT_INFO_STATE_ARCHIVE,
calculateArchiveKeyNoContextMinSuffix(startKeyHash.toArrayUnsafe()),
calculateArchiveKeyNoContextMaxSuffix(endKeyHash.toArrayUnsafe()))
.map(e -> Bytes.of(calculateArchiveKeyNoContextMaxSuffix(trimSuffix(e.getKey()))))
Expand All @@ -190,7 +190,11 @@ protected Stream<Pair<Bytes32, Bytes>> accountsToPairStream(
new Pair<>(
Bytes32.wrap(trimSuffix(e.toArrayUnsafe())),
Bytes.of(
storage.getNearestBefore(ACCOUNT_INFO_STATE, e).get().value().get())));
storage
.getNearestBefore(ACCOUNT_INFO_STATE_ARCHIVE, e)
.get()
.value()
.get())));
return stream;
}

Expand All @@ -200,7 +204,7 @@ protected Stream<Pair<Bytes32, Bytes>> accountsToPairStream(
final Stream<Pair<Bytes32, Bytes>> stream =
storage
.streamFromKey(
ACCOUNT_INFO_STATE,
ACCOUNT_INFO_STATE_ARCHIVE,
calculateArchiveKeyNoContextMinSuffix(startKeyHash.toArrayUnsafe()))
.map(e -> Bytes.of(calculateArchiveKeyNoContextMaxSuffix(trimSuffix(e.getKey()))))
.distinct()
Expand All @@ -209,7 +213,11 @@ protected Stream<Pair<Bytes32, Bytes>> accountsToPairStream(
new Pair<Bytes32, Bytes>(
Bytes32.wrap(trimSuffix(e.toArrayUnsafe())),
Bytes.of(
storage.getNearestBefore(ACCOUNT_INFO_STATE, e).get().value().get())));
storage
.getNearestBefore(ACCOUNT_INFO_STATE_ARCHIVE, e)
.get()
.value()
.get())));
return stream;
}

Expand All @@ -221,7 +229,7 @@ protected Stream<Pair<Bytes32, Bytes>> storageToPairStream(
final Function<Bytes, Bytes> valueMapper) {
return storage
.streamFromKey(
ACCOUNT_STORAGE_STORAGE,
ACCOUNT_STORAGE_ARCHIVE,
calculateArchiveKeyNoContextMinSuffix(
calculateNaturalSlotKey(accountHash, Hash.wrap(Bytes32.wrap(startKeyHash)))))
.map(e -> Bytes.of(calculateArchiveKeyNoContextMaxSuffix(trimSuffix(e.getKey()))))
Expand All @@ -234,7 +242,7 @@ protected Stream<Pair<Bytes32, Bytes>> storageToPairStream(
valueMapper.apply(
Bytes.of(
storage
.getNearestBefore(ACCOUNT_STORAGE_STORAGE, key)
.getNearestBefore(ACCOUNT_STORAGE_ARCHIVE, key)
.get()
.value()
.get())
Expand All @@ -250,7 +258,7 @@ protected Stream<Pair<Bytes32, Bytes>> storageToPairStream(
final Function<Bytes, Bytes> valueMapper) {
return storage
.streamFromKey(
ACCOUNT_STORAGE_STORAGE,
ACCOUNT_STORAGE_ARCHIVE,
calculateArchiveKeyNoContextMinSuffix(
calculateNaturalSlotKey(accountHash, Hash.wrap(Bytes32.wrap(startKeyHash)))),
calculateArchiveKeyNoContextMaxSuffix(
Expand All @@ -265,7 +273,7 @@ protected Stream<Pair<Bytes32, Bytes>> storageToPairStream(
valueMapper.apply(
Bytes.of(
storage
.getNearestBefore(ACCOUNT_STORAGE_STORAGE, key)
.getNearestBefore(ACCOUNT_STORAGE_ARCHIVE, key)
.get()
.value()
.get())
Expand All @@ -287,7 +295,7 @@ public void putFlatAccount(
calculateArchiveKeyWithMinSuffix(
getStateArchiveContextForWrite(storage).get(), accountHash.getBytes().toArrayUnsafe());

transaction.put(ACCOUNT_INFO_STATE, keySuffixed, accountValue.toArrayUnsafe());
transaction.put(ACCOUNT_INFO_STATE_ARCHIVE, keySuffixed, accountValue.toArrayUnsafe());
}

@Override
Expand All @@ -301,7 +309,7 @@ public void removeFlatAccount(
calculateArchiveKeyWithMinSuffix(
getStateArchiveContextForWrite(storage).get(), accountHash.getBytes().toArrayUnsafe());

transaction.put(ACCOUNT_INFO_STATE, keySuffixed, DELETED_ACCOUNT_VALUE);
transaction.put(ACCOUNT_INFO_STATE_ARCHIVE, keySuffixed, DELETED_ACCOUNT_VALUE);
}

private byte[] trimSuffix(final byte[] suffixedAddress) {
Expand Down Expand Up @@ -332,7 +340,7 @@ public Optional<Bytes> getFlatStorageValueByStorageSlotKey(
// Find the nearest storage for this address, slot key hash, and block context
Optional<SegmentedKeyValueStorage.NearestKeyValue> nearestStorage =
storage
.getNearestBefore(ACCOUNT_STORAGE_STORAGE, keyNearest)
.getNearestBefore(ACCOUNT_STORAGE_ARCHIVE, keyNearest)
.filter(
found -> Bytes.of(naturalKey).commonPrefixLength(found.key()) >= naturalKey.length);

Expand All @@ -341,7 +349,7 @@ public Optional<Bytes> getFlatStorageValueByStorageSlotKey(
// Check the archived storage as old state is moved out of the primary DB segment
storageFound =
storage
.getNearestBefore(ACCOUNT_STORAGE_ARCHIVE, keyNearest)
.getNearestBefore(ACCOUNT_STORAGE_FREEZER, keyNearest)
// don't return accounts that do not have a matching account hash
.filter(
found ->
Expand Down Expand Up @@ -389,7 +397,7 @@ public void putFlatAccountStorageValueByStorageSlotHash(
byte[] keyNearest =
calculateArchiveKeyWithMinSuffix(getStateArchiveContextForWrite(storage).get(), naturalKey);

transaction.put(ACCOUNT_STORAGE_STORAGE, keyNearest, storageValue.toArrayUnsafe());
transaction.put(ACCOUNT_STORAGE_ARCHIVE, keyNearest, storageValue.toArrayUnsafe());
}

/*
Expand All @@ -408,7 +416,7 @@ public void removeFlatAccountStorageValueByStorageSlotHash(
byte[] keySuffixed =
calculateArchiveKeyWithMinSuffix(getStateArchiveContextForWrite(storage).get(), naturalKey);

transaction.put(ACCOUNT_STORAGE_STORAGE, keySuffixed, DELETED_STORAGE_VALUE);
transaction.put(ACCOUNT_STORAGE_ARCHIVE, keySuffixed, DELETED_STORAGE_VALUE);
}

public static byte[] calculateNaturalSlotKey(final Hash accountHash, final Hash slotHash) {
Expand All @@ -433,6 +441,27 @@ public static Bytes calculateArchiveKeyWithMaxSuffix(
return Bytes.of(calculateArchiveKeyWithSuffix(context, naturalKey, MAX_BLOCK_SUFFIX));
}

@Override
public void clearAll(final SegmentedKeyValueStorage storage) {
clearArchiveSegments(storage);
// Then call parent to clear other segments
super.clearAll(storage);
}

@Override
public void resetOnResync(final SegmentedKeyValueStorage storage) {
clearArchiveSegments(storage);
// Then call parent to reset other segments
super.resetOnResync(storage);
}

private static void clearArchiveSegments(final SegmentedKeyValueStorage storage) {
storage.clear(ACCOUNT_INFO_STATE_ARCHIVE);
storage.clear(ACCOUNT_STORAGE_ARCHIVE);
storage.clear(ACCOUNT_INFO_STATE_FREEZER);
storage.clear(ACCOUNT_STORAGE_FREEZER);
}
Comment thread
jframe marked this conversation as resolved.

// TODO JF: move this out of this class so can be used with ArchiveCodeStorageStrategy without
// being static
public static byte[] calculateArchiveKeyWithSuffix(
Expand Down
26 changes: 14 additions & 12 deletions ...edger/besu/ethereum/trie/pathbased/common/storage/PathBasedWorldStateKeyValueStorage.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,9 @@

import static org.hyperledger.besu.ethereum.storage.keyvalue.KeyValueSegmentIdentifier.ACCOUNT_INFO_STATE;
import static org.hyperledger.besu.ethereum.storage.keyvalue.KeyValueSegmentIdentifier.ACCOUNT_INFO_STATE_ARCHIVE;
import static org.hyperledger.besu.ethereum.storage.keyvalue.KeyValueSegmentIdentifier.ACCOUNT_INFO_STATE_FREEZER;
import static org.hyperledger.besu.ethereum.storage.keyvalue.KeyValueSegmentIdentifier.ACCOUNT_STORAGE_ARCHIVE;
import static org.hyperledger.besu.ethereum.storage.keyvalue.KeyValueSegmentIdentifier.ACCOUNT_STORAGE_FREEZER;
import static org.hyperledger.besu.ethereum.storage.keyvalue.KeyValueSegmentIdentifier.ACCOUNT_STORAGE_STORAGE;
import static org.hyperledger.besu.ethereum.storage.keyvalue.KeyValueSegmentIdentifier.CODE_STORAGE;
import static org.hyperledger.besu.ethereum.storage.keyvalue.KeyValueSegmentIdentifier.TRIE_BRANCH_STORAGE;
Expand Down Expand Up @@ -216,9 +218,9 @@ public boolean pruneTrieLog(final Hash blockHash) {
}

/**
* Move old account state from the primary DB segment to the archive segment that will only be
* used for historic state queries. This prevents performance degradation over time for writes to
* the primary DB segments.
* Move old account state from the primary DB archive segment to the archive freezer segment that
* will only be used for historic state queries. This prevents performance degradation over time
* for writes to the primary archive DB segments.
*
* @param previousBlockHeader the block header for the previous block, used to get the "nearest
* before" state
Expand All @@ -243,7 +245,7 @@ public int archivePreviousAccountState(
// Move all entries that match this address hash to the archive DB segment
while ((nextMatch =
composedWorldStateStorage
.getNearestBefore(ACCOUNT_INFO_STATE, previousKey)
.getNearestBefore(ACCOUNT_INFO_STATE_ARCHIVE, previousKey)
.filter(
found ->
found.value().isPresent()
Expand All @@ -254,8 +256,8 @@ public int archivePreviousAccountState(
.forEach(
(nearestKey) -> {
moveDBEntry(
ACCOUNT_INFO_STATE,
ACCOUNT_INFO_STATE_ARCHIVE,
ACCOUNT_INFO_STATE_FREEZER,
nearestKey.key().toArrayUnsafe(),
nearestKey.value().get());
archivedStateCount.getAndIncrement();
Expand Down Expand Up @@ -287,9 +289,9 @@ public int archivePreviousAccountState(
}

/**
* Move old storage state from the primary DB segment to the archive segment that will only be
* used for historic state queries. This prevents performance degradation over time for writes to
* the primary DB segments.
* Move old storage state from the primary archive DB segment to the archive freezer segment that
* will only be used for historic state queries. This prevents performance degradation over time
* for writes to the primary DB segments.
*
* @param previousBlockHeader the block header for the previous block, used to get the "nearest
* before" state
Expand All @@ -315,7 +317,7 @@ public int archivePreviousStorageState(
// to the archive DB segment
while ((nextMatch =
composedWorldStateStorage
.getNearestBefore(ACCOUNT_STORAGE_STORAGE, previousKey)
.getNearestBefore(ACCOUNT_STORAGE_ARCHIVE, previousKey)
.filter(
found ->
found.value().isPresent()
Expand All @@ -338,8 +340,8 @@ public int archivePreviousStorageState(
.log();
}
moveDBEntry(
ACCOUNT_STORAGE_STORAGE,
ACCOUNT_STORAGE_ARCHIVE,
ACCOUNT_STORAGE_FREEZER,
nearestKey.key().toArrayUnsafe(),
nearestKey.value().get());
archivedStorageCount.getAndIncrement();
Expand Down Expand Up @@ -395,15 +397,15 @@ private void moveDBEntry(

public Optional<Long> getLatestArchivedBlock() {
return composedWorldStateStorage
.get(ACCOUNT_INFO_STATE_ARCHIVE, ARCHIVED_BLOCKS)
.get(ACCOUNT_INFO_STATE_FREEZER, ARCHIVED_BLOCKS)
.map(Bytes::wrap)
.map(Bytes::toLong);
}

public void setLatestArchivedBlock(final Long blockNumber) {
SegmentedKeyValueStorageTransaction tx = composedWorldStateStorage.startTransaction();
tx.put(
ACCOUNT_INFO_STATE_ARCHIVE,
ACCOUNT_INFO_STATE_FREEZER,
ARCHIVED_BLOCKS,
Bytes.ofUnsignedLong(blockNumber).toArrayUnsafe());
tx.commit();
Expand Down
Loading
Loading