IntermediateRoot: add flag for threshold to update concurrently #453
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Francesco4203
force-pushed
the
fix/concurrent-update-2
branch
from
70ae86c
to
b0f24b0
Compare
minh-bq
reviewed
May 21, 2024
Divide the root updating of stateObjects into goroutines if number of stateObjects is at least the threshold -> results in 1.55 times faster statedb_test.go/TestIntermediateUpdateConcurrently: add test to check if the states after processed with both options are identical
statedb/intermediateRoot: update options for concurrent: get changes that are not made yet to run concurrently state_object/updateTrie: add option for concurrency: store and return data waiting for update
Francesco4203
force-pushed
the
fix/concurrent-update-2
branch
from
b0f24b0
to
08fd7c3
Compare
…n be run concurrently
minh-bq
reviewed
May 27, 2024
core/state/state_object.go
Outdated
| @@ -319,7 +319,7 @@ func (s *stateObject) finalise(prefetch bool) { | |||
|
|
|||
| // updateTrie writes cached storage modifications into the object's storage trie. | |||
| // It will return nil if the trie has not been loaded and no changes have been made | |||
| func (s *stateObject) updateTrie(db Database) Trie { | |||
| func (s *stateObject) updateTrie(db Database, concurrent bool) Trie { | |||
There was a problem hiding this comment.
I think change this variable to skipTrieUpdate is more reasonable
core/state/state_object.go
Outdated
|
|
||
| if (value == common.Hash{}) { | ||
| s.setError(tr.TryDelete(key[:])) | ||
| s.db.StorageDeleted += 1 |
There was a problem hiding this comment.
This is shared resource, not safe to update concurrently. Moreover, this is updated in updateTrie already
core/state/state_object.go
Outdated
| } else { | ||
| v, _ = rlp.EncodeToBytes(common.TrimLeftZeroes(value[:])) | ||
| s.setError(tr.TryUpdate(key[:], v)) | ||
| s.db.StorageUpdated += 1 |
core/state/statedb.go
Outdated
| @@ -855,11 +863,52 @@ func (s *StateDB) IntermediateRoot(deleteEmptyObjects bool) common.Hash { | |||
| // the account prefetcher. Instead, let's process all the storage updates | |||
| // first, giving the account prefeches just a few more milliseconds of time | |||
| // to pull useful data from disk. | |||
|
|
|||
| // ---------------------------------------------- DEBUGGING ------------------------------------------------------------------------------------------- | |||
core/state/statedb.go
Outdated
| for addr := range s.stateObjectsPending { | ||
| if obj := s.stateObjects[addr]; !obj.deleted { | ||
| obj.updateRoot(s.db) | ||
| updateObjs = append(updateObjs, obj) | ||
| obj.updateTrie(s.db, true) |
There was a problem hiding this comment.
We should use ConcurrentUpdateThreshold config to determine if we need to update trie concurrently or not
core/state/statedb.go
Outdated
| } | ||
| } | ||
| min := func(a, b int) int { |
There was a problem hiding this comment.
With the new way of calculating number of objects per routine, this function is only called once, so I think removing this function and using if for that one is better
core/state/state_object.go
Outdated
| @@ -332,26 +332,33 @@ func (s *stateObject) updateTrie(db Database) Trie { | |||
| // The snapshot storage map for the object | |||
| var storage map[common.Hash][]byte | |||
| // Insert all the pending updates into the trie | |||
| tr := s.getTrie(db) | |||
| var tr Trie = nil | |||
There was a problem hiding this comment.
nit: don't need to initialize to nil here, the default value is nil already
statedb.go/StateDB struct: remove lock since it will not be used for concurrent update statedb.go/IntermediateRoot: check ConcurrentUpdateThreshold for choosing between sequential and concurrent update state_object.go: change "concurrent" variable to a more reasonable name "skipTrieUpdate" flags.go/MakeChain: add flag when creating a chain
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Divide the root updating of stateObjects into goroutines if number of stateObjects is at least the threshold