-
Notifications
You must be signed in to change notification settings - Fork 852
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Improve genesis state performance at startup #6977
Improve genesis state performance at startup #6977
Conversation
Signed-off-by: Fabio Di Fabio <[email protected]>
Signed-off-by: Fabio Di Fabio <[email protected]>
Signed-off-by: Fabio Di Fabio <[email protected]>
Signed-off-by: Fabio Di Fabio <[email protected]>
Signed-off-by: Fabio Di Fabio <[email protected]>
Hey, I tested the current code; here are some records:
Before your refactor, the first start of Besu needed only 7 mins and 186 GB of memory allocated. The test record is in the jfr record file; you can load it directly using IntelliJ IDEA: https://github.com/lyfsn/besu-starter/blob/main/g-1.jfr |
Signed-off-by: Fabio Di Fabio <[email protected]>
Signed-off-by: Fabio Di Fabio <[email protected]>
Signed-off-by: Fabio Di Fabio <[email protected]>
Signed-off-by: Fabio Di Fabio <[email protected]>
…u into refactor-genesis-file-options
Signed-off-by: Fabio Di Fabio <[email protected]>
Signed-off-by: Fabio Di Fabio <[email protected]>
08ab687
to
0889ee4
Compare
# Conflicts: # acceptance-tests/dsl/src/main/java/org/hyperledger/besu/tests/acceptance/dsl/node/ThreadBesuNodeRunner.java # besu/src/main/java/org/hyperledger/besu/cli/BesuCommand.java # besu/src/main/java/org/hyperledger/besu/cli/config/EthNetworkConfig.java # besu/src/main/java/org/hyperledger/besu/controller/BesuController.java # besu/src/main/java/org/hyperledger/besu/controller/BesuControllerBuilder.java # besu/src/main/java/org/hyperledger/besu/controller/TransitionBesuControllerBuilder.java # besu/src/test/java/org/hyperledger/besu/PrivacyReorgTest.java # besu/src/test/java/org/hyperledger/besu/RunnerTest.java # besu/src/test/java/org/hyperledger/besu/cli/BesuCommandTest.java # besu/src/test/java/org/hyperledger/besu/cli/CascadingDefaultProviderTest.java # besu/src/test/java/org/hyperledger/besu/cli/config/EthNetworkConfigTest.java # besu/src/test/java/org/hyperledger/besu/controller/BesuControllerTest.java # config/src/main/java/org/hyperledger/besu/config/GenesisConfigFile.java # config/src/test/java/org/hyperledger/besu/config/JsonUtilTest.java # consensus/merge/src/test/java/org/hyperledger/besu/consensus/merge/blockcreation/MergeGenesisConfigHelper.java # consensus/qbft/src/integration-test/java/org/hyperledger/besu/consensus/qbft/support/TestContextBuilder.java # ethereum/core/src/test/java/org/hyperledger/besu/ethereum/trie/diffbased/bonsai/AbstractIsolationTests.java
9ca104c
to
b3a8978
Compare
# Conflicts: # config/src/main/java/org/hyperledger/besu/config/JsonUtil.java
829b437
to
07ab31c
Compare
Signed-off-by: Fabio Di Fabio <[email protected]>
07ab31c
to
a662895
Compare
Signed-off-by: Fabio Di Fabio <[email protected]>
Signed-off-by: ahamlat <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
With this PR, starting with a genesis file with more than 1 GiB took 4 minutes and 30 seconds with -Xmx16g.
Without this PR, even with -Xmx16g, the node got killed after 8 minutes.
PR description
This PR takes inspiration from #6952 to improve Besu starts after the first one, specifically when the database is present, when
genesis-state-hash-cache-enabled=true
, since it completely avoid to reprocess the genesis state, and in case of large genesis files, with may allocations, the startup time drops from many minutes to few seconds, and in any case the genesis allocations are not kept line in memory.The core of the change is to make the loading of the genesis allocations lazy, so they are only parsed from file when needed, and switching to a streaming parser so they are not all loaded and kept in memory.
Fixed Issue(s)
Thanks for sending a pull request! Have you done the following?
doc-change-required
label to this PR if updates are required.Locally, you can run these tests to catch failures early:
./gradlew build
./gradlew acceptanceTest
./gradlew integrationTest
./gradlew ethereum:referenceTests:referenceTests