cmd/evm: make batched state-test execution possible#27318
Merged
Conversation
MariusVanDerWijden
approved these changes
May 22, 2023
Member
MariusVanDerWijden
left a comment
There was a problem hiding this comment.
LGTM a great feature which will speed up the fuzzing significantly
AskAlexSharov
pushed a commit
to erigontech/erigon
that referenced
this pull request
May 29, 2023
This implements batched state-test exectution, similar to ethereum/go-ethereum#27318 . Some speedtests, executing a state-test twice on current master takes ~4-5 seconds, and scales linerarly. ``` Doing 2 execs old style real 0m8.185s user 0m8.081s sys 0m0.110s ``` Doing `100` executions on this PR -- a few seconds of ramp-up time, but very quick execution after that : ``` Doing 100 execs v2 real 0m5.009s user 0m4.560s sys 0m0.508s ``` I also tested a version where I moved the db instantiation into the top callsite, with the `MustOpen` and `.Close` only performed once, instead of `100` times -- however, I noticed no additional speed gains from doing so (my branch `batched_evm_v2`). Therefore, I suspect that the slowdowns comes not from the db, but the kzg library initialization.
devopsbo3
pushed a commit
to HorizenOfficial/go-ethereum
that referenced
this pull request
Nov 10, 2023
implements the ability to run several state-tests in one instance. By not providing a statetest path to the `evm statetest` command, the path(s) will instead be read from `stdin`.
devopsbo3
added a commit
to HorizenOfficial/go-ethereum
that referenced
this pull request
Nov 10, 2023
devopsbo3
added a commit
to HorizenOfficial/go-ethereum
that referenced
this pull request
Nov 10, 2023
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR implements a feature that is already present in both nethermind and besu: the ability to run several state-tests in one instance.
For state-tests that are reasonably small, this makes a huge difference. Here's 1000 executions of a state-test which has a run-length of merely
28steps.Nearly one minute versus
<2seconds. When running it in a fuzzing-setting, the speedups is less marked, somewhere in the1x-2xregion.If the kzg stuff now has a static overhead, as suggested in #27258, this PR would definitely help (the fuzzer). Perhaps something similar can be done for
evm t8ntoo.