Skip to content

Call operation performance optimizations #5537

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
shemnon merged 13 commits into from
Aug 8, 2023
Merged

Call operation performance optimizations #5537

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
d43f395
Undoable Collections
shemnon May 10, 2023
20a7618
Merge branch 'main' of github.com:hyperledger/besu into undoMaps
shemnon May 30, 2023
7264d82
Merge branch 'main' of github.com:hyperledger/besu into undoMaps
shemnon Jun 5, 2023
569f314
Call Operation performance imporvements
shemnon Jun 5, 2023
fd91e18
spotless
shemnon Jun 5, 2023
57cae28
spotless, copyright, fix unit and integration tests.
shemnon Jun 7, 2023
9ad511c
static
shemnon Jun 7, 2023
3d998a9
merge main
shemnon Jul 28, 2023
18936e1
spotless
shemnon Jul 28, 2023
8feef41
more address hashes
shemnon Jul 31, 2023
fb94c19
compiler errors
shemnon Jul 31, 2023
7d52c38
spotless
shemnon Aug 1, 2023
3858bbb
Merge branch 'main' into call-optimization
shemnon Aug 7, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions datatypes/src/main/java/org/hyperledger/besu/datatypes/Address.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,12 @@
import org.hyperledger.besu.ethereum.rlp.RLPException;
import org.hyperledger.besu.ethereum.rlp.RLPInput;

import java.util.concurrent.ExecutionException;

import com.fasterxml.jackson.annotation.JsonCreator;
import com.google.common.cache.CacheBuilder;
import com.google.common.cache.CacheLoader;
import com.google.common.cache.LoadingCache;
import org.apache.tuweni.bytes.Bytes;
import org.apache.tuweni.bytes.Bytes32;
import org.apache.tuweni.bytes.DelegatingBytes;
Expand Down Expand Up @@ -75,6 +80,18 @@ public class Address extends DelegatingBytes {
/** The constant ZERO. */
public static final Address ZERO = Address.fromHexString("0x0");

static LoadingCache<Address, Hash> hashCache =
Copy link
Contributor

@matkt matkt Jun 6, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we don't want to add a limit for this cache ?

Copy link
Contributor Author

@shemnon shemnon Jun 7, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What limit would you suggest?

Copy link
Contributor

@matkt matkt Jun 7, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would say a maximum size. Maybe we should see how much on average we load hash by block and got a number. But with a cache without max seems dangerous to me . Or else there is a default limit?

CacheBuilder.newBuilder()
.maximumSize(4000)
// .weakKeys() // unless we "intern" all addresses we cannot use weak or soft keys.
.build(
new CacheLoader<>() {
@Override
public Hash load(final Address key) {
return Hash.hash(key);
}
});

/**
* Instantiates a new Address.
*
Expand Down Expand Up @@ -237,4 +254,17 @@ public static Address privateContractAddress(
out.endList();
})));
}

/**
* Returns the hash of the address. Backed by a cache for performance reasons.
*
* @return the hash of the address.
*/
public Hash addressHash() {
try {
return hashCache.get(this);
} catch (ExecutionException e) {
return Hash.hash(this);
}
}
}
10 changes: 2 additions & 8 deletions ...re/src/jmh/java/org/hyperledger/besu/ethereum/vm/operations/OperationBenchmarkHelper.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -112,25 +112,19 @@ public MessageFrame createMessageFrame() {

public MessageFrame.Builder createMessageFrameBuilder() {
return MessageFrame.builder()
.parentMessageFrame(messageFrame)
.type(MessageFrame.Type.MESSAGE_CALL)
.messageFrameStack(messageFrame.getMessageFrameStack())
.worldUpdater(messageFrame.getWorldUpdater())
.initialGas(messageFrame.getRemainingGas())
.address(messageFrame.getContractAddress())
.originator(messageFrame.getOriginatorAddress())
.contract(messageFrame.getRecipientAddress())
.gasPrice(messageFrame.getGasPrice())
.inputData(messageFrame.getInputData())
.sender(messageFrame.getSenderAddress())
.value(messageFrame.getValue())
.apparentValue(messageFrame.getApparentValue())
.code(messageFrame.getCode())
.blockValues(messageFrame.getBlockValues())
.depth(messageFrame.getMessageStackDepth())
.isStatic(messageFrame.isStatic())
.completer(messageFrame -> {})
.miningBeneficiary(messageFrame.getMiningBeneficiary())
.maxStackSize(messageFrame.getMaxStackSize());
.completer(frame -> {});
}

public void cleanUp() throws IOException {
Expand Down
4 changes: 2 additions & 2 deletions ethereum/core/src/main/java/org/hyperledger/besu/ethereum/bonsai/BonsaiAccount.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ public BonsaiAccount(
this(
context,
address,
Hash.hash(address),
address.addressHash(),
stateTrieAccount.getNonce(),
stateTrieAccount.getBalance(),
stateTrieAccount.getStorageRoot(),
Expand Down Expand Up @@ -142,7 +142,7 @@ public static BonsaiAccount fromRLP(
in.leaveList();

return new BonsaiAccount(
context, address, Hash.hash(address), nonce, balance, storageRoot, codeHash, mutable);
context, address, address.addressHash(), nonce, balance, storageRoot, codeHash, mutable);
}

@Override
Expand Down
14 changes: 7 additions & 7 deletions ...eum/core/src/main/java/org/hyperledger/besu/ethereum/bonsai/BonsaiWorldStateProvider.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -105,11 +105,11 @@ public BonsaiWorldStateProvider(
this.persistedState = new BonsaiWorldState(this, worldStateStorage);
this.cachedMerkleTrieLoader = cachedMerkleTrieLoader;
blockchain
.getBlockHeader(persistedState.worldStateBlockHash)
.getBlockHeader(persistedState.getWorldStateBlockHash())
.ifPresent(
blockHeader ->
this.trieLogManager.addCachedLayer(
blockHeader, persistedState.worldStateRootHash, persistedState));
blockHeader, persistedState.getWorldStateRootHash(), persistedState));
}

@VisibleForTesting
Expand All @@ -124,11 +124,11 @@ public BonsaiWorldStateProvider(
this.persistedState = new BonsaiWorldState(this, worldStateStorage);
this.cachedMerkleTrieLoader = cachedMerkleTrieLoader;
blockchain
.getBlockHeader(persistedState.worldStateBlockHash)
.getBlockHeader(persistedState.getWorldStateBlockHash())
.ifPresent(
blockHeader ->
this.trieLogManager.addCachedLayer(
blockHeader, persistedState.worldStateRootHash, persistedState));
blockHeader, persistedState.getWorldStateRootHash(), persistedState));
}

@Override
Expand Down Expand Up @@ -300,7 +300,7 @@ public MutableWorldState getMutable() {
public void prepareStateHealing(final Address address, final Bytes location) {
final Set<Bytes> keysToDelete = new HashSet<>();
final BonsaiWorldStateKeyValueStorage.BonsaiUpdater updater = worldStateStorage.updater();
final Hash accountHash = Hash.hash(address);
final Hash accountHash = address.addressHash();
final StoredMerklePatriciaTrie<Bytes, Bytes> accountTrie =
new StoredMerklePatriciaTrie<>(
(l, h) -> {
Expand All @@ -310,7 +310,7 @@ public void prepareStateHealing(final Address address, final Bytes location) {
}
return node;
},
persistedState.worldStateRootHash,
persistedState.getWorldStateRootHash(),
Function.identity(),
Function.identity());
try {
Expand Down Expand Up @@ -359,7 +359,7 @@ public void resetArchiveStateTo(final BlockHeader blockHeader) {
persistedState.resetWorldStateTo(blockHeader);
this.trieLogManager.reset();
this.trieLogManager.addCachedLayer(
blockHeader, persistedState.worldStateRootHash, persistedState);
blockHeader, persistedState.getWorldStateRootHash(), persistedState);
}

@Override
Expand Down
9 changes: 4 additions & 5 deletions ...core/src/main/java/org/hyperledger/besu/ethereum/bonsai/cache/CachedMerkleTrieLoader.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,9 +52,8 @@ public CachedMerkleTrieLoader(final ObservableMetricsSystem metricsSystem) {
CacheMetricsCollector cacheMetrics = new CacheMetricsCollector();
cacheMetrics.addCache("accountsNodes", accountNodes);
cacheMetrics.addCache("storageNodes", storageNodes);
if (metricsSystem instanceof PrometheusMetricsSystem)
((PrometheusMetricsSystem) metricsSystem)
.addCollector(BesuMetricCategory.BLOCKCHAIN, () -> cacheMetrics);
if (metricsSystem instanceof PrometheusMetricsSystem prometheusMetricsSystem)
prometheusMetricsSystem.addCollector(BesuMetricCategory.BLOCKCHAIN, () -> cacheMetrics);
}

public void preLoadAccount(
Expand Down Expand Up @@ -82,7 +81,7 @@ public void cacheAccountNodes(
worldStateRootHash,
Function.identity(),
Function.identity());
accountTrie.get(Hash.hash(account));
accountTrie.get(account.addressHash());
} catch (MerkleTrieException e) {
// ignore exception for the cache
} finally {
Expand All @@ -102,7 +101,7 @@ public void cacheStorageNodes(
final BonsaiWorldStateKeyValueStorage worldStateStorage,
final Address account,
final StorageSlotKey slotKey) {
final Hash accountHash = Hash.hash(account);
final Hash accountHash = account.addressHash();
final long storageSubscriberId = worldStateStorage.subscribe(this);
try {
worldStateStorage
Expand Down
15 changes: 6 additions & 9 deletions ...e/src/main/java/org/hyperledger/besu/ethereum/bonsai/cache/CachedWorldStorageManager.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,6 @@
import java.util.Optional;
import java.util.concurrent.ConcurrentHashMap;
import java.util.function.Function;
import java.util.stream.Collectors;
import java.util.stream.LongStream;
import java.util.stream.Stream;

Expand Down Expand Up @@ -106,7 +105,7 @@ public synchronized void addCachedLayer(
.get()
.updateWorldStateStorage(
new BonsaiSnapshotWorldStateKeyValueStorage(
forWorldState.worldStateStorage, metricsSystem));
forWorldState.getWorldStateStorage(), metricsSystem));
}
} else {
LOG.atDebug()
Expand All @@ -120,7 +119,7 @@ public synchronized void addCachedLayer(
new CachedBonsaiWorldView(
blockHeader,
new BonsaiSnapshotWorldStateKeyValueStorage(
forWorldState.worldStateStorage, metricsSystem)));
forWorldState.getWorldStateStorage(), metricsSystem)));
} else {
// otherwise, add the layer to the cache
cachedWorldStatesByHash.put(
Expand Down Expand Up @@ -257,14 +256,12 @@ protected TrieLogFactory setupTrieLogFactory(final BesuContext pluginContext) {
TrieLogProvider getTrieLogProvider() {
return new TrieLogProvider() {
@Override
public <T extends TrieLog.LogTuple<?>> Optional<TrieLog> getTrieLogLayer(
final Hash blockHash) {
public Optional<TrieLog> getTrieLogLayer(final Hash blockHash) {
return CachedWorldStorageManager.this.getTrieLogLayer(blockHash);
}

@Override
public <T extends TrieLog.LogTuple<?>> Optional<TrieLog> getTrieLogLayer(
final long blockNumber) {
public Optional<TrieLog> getTrieLogLayer(final long blockNumber) {
return CachedWorldStorageManager.this
.blockchain
.getBlockHeader(blockNumber)
Expand All @@ -273,7 +270,7 @@ public <T extends TrieLog.LogTuple<?>> Optional<TrieLog> getTrieLogLayer(
}

@Override
public <T extends TrieLog.LogTuple<?>> List<TrieLogRangeTuple> getTrieLogsByRange(
public List<TrieLogRangeTuple> getTrieLogsByRange(
final long fromBlockNumber, final long toBlockNumber) {
return rangeAsStream(fromBlockNumber, toBlockNumber)
.map(blockchain::getBlockHeader)
Expand All @@ -289,7 +286,7 @@ public <T extends TrieLog.LogTuple<?>> List<TrieLogRangeTuple> getTrieLogsByRang
header.getBlockHash(), header.getNumber(), layer))))
.filter(Optional::isPresent)
.map(Optional::get)
.collect(Collectors.toList());
.toList();
}

Stream<Long> rangeAsStream(final long fromBlockNumber, final long toBlockNumber) {
Expand Down
Loading